Appending data to existing data stored in a dispersed storage network

ABSTRACT

A method begins by a processing module receiving a request to store data in dispersed storage network (DSN) memory and determining whether the data is to be appended to existing data. When the data is to be appended, the method continues with the processing module encoding, using an append dispersed storage error coding function, the data to produce a set of encoded append data slices, generating a set of append commands, wherein an append command of the set of append commands includes an encoded append data slice of the set of encoded append data slices and identity of one of a set of dispersed storage (DS) units, and outputting at least a write threshold number of the set of append commands to at least a write threshold number of the set of DS units.

CROSS REFERENCE TO RELATED PATENTS

The present U.S. Utility Patent Application claims priority pursuant to35 U.S.C. §119(e) to U.S. Provisional Application No. 61/470,524,entitled “Encoding Data Stored in a Dispersed Storage Network,” filedApr. 1, 2011, pending, which is incorporated herein by reference in itsentirety and made part of the present U.S. Utility Patent Applicationfor all purposes.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

NOT APPLICABLE

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

NOT APPLICABLE

BACKGROUND OF THE INVENTION

1. Technical Field of the Invention

This invention relates generally to computing systems and moreparticularly to data storage solutions within such computing systems.

2. Description of Related Art

Computers are known to communicate, process, and store data. Suchcomputers range from wireless smart phones to data centers that supportmillions of web searches, stock trades, or on-line purchases every day.In general, a computing system generates data and/or manipulates datafrom one form into another. For instance, an image sensor of thecomputing system generates raw picture data and, using an imagecompression program (e.g., JPEG, MPEG, etc.), the computing systemmanipulates the raw picture data into a standardized compressed image.

With continued advances in processing speed and communication speed,computers are capable of processing real time multimedia data forapplications ranging from simple voice communications to streaming highdefinition video. As such, general-purpose information appliances arereplacing purpose-built communications devices (e.g., a telephone). Forexample, smart phones can support telephony communications but they arealso capable of text messaging and accessing the Internet to performfunctions including email, web browsing, remote applications access, andmedia communications (e.g., telephony voice, image transfer, musicfiles, video files, real time video streaming. etc.).

Each type of computer is constructed and operates in accordance with oneor more communication, processing, and storage standards. As a result ofstandardization and with advances in technology, more and moreinformation content is being converted into digital formats. Forexample, more digital cameras are now being sold than film cameras, thusproducing more digital pictures. As another example, web-basedprogramming is becoming an alternative to over the air televisionbroadcasts and/or cable broadcasts. As further examples, papers, books,video entertainment, home video, etc. are now being stored digitally,which increases the demand on the storage function of computers.

A typical computer storage system includes one or more memory devicesaligned with the needs of the various operational aspects of thecomputer's processing and communication functions. Generally, theimmediacy of access dictates what type of memory device is used. Forexample, random access memory (RAM) memory can be accessed in any randomorder with a constant response time, thus it is typically used for cachememory and main memory. By contrast, memory device technologies thatrequire physical movement such as magnetic disks, tapes, and opticaldiscs, have a variable response time as the physical movement can takelonger than the data transfer, thus they are typically used forsecondary memory (e.g., hard drive, backup memory, etc.).

A computer's storage system will be compliant with one or more computerstorage standards that include, but are not limited to, network filesystem (NFS), flash file system (FFS), disk file system (DFS), smallcomputer system interface (SCSI), internet small computer systeminterface (iSCSI), file transfer protocol (FTP), and web-baseddistributed authoring and versioning (WebDAV). These standards specifythe data storage format (e.g., files, data objects, data blocks,directories, etc.) and interfacing between the computer's processingfunction and its storage system, which is a primary function of thecomputer's memory controller.

Despite the standardization of the computer and its storage system,memory devices fail; especially commercial grade memory devices thatutilize technologies incorporating physical movement (e.g., a discdrive). For example, it is fairly common for a disc drive to routinelysuffer from bit level corruption and to completely fail after threeyears of use. One solution is to a higher-grade disc drive, which addssignificant cost to a computer.

Another solution is to utilize multiple levels of redundant disc drivesto replicate the data into two or more copies. One such redundant driveapproach is called redundant array of independent discs (RAID). In aRAID device, a RAID controller adds parity data to the original databefore storing it across the array. The parity data is calculated fromthe original data such that the failure of a disc will not result in theloss of the original data. For example, RAID 5 uses three discs toprotect data from the failure of a single disc. The parity data, andassociated redundancy overhead data, reduces the storage capacity ofthree independent discs by one third (e.g., n−1=capacity). RAID 6 canrecover from a loss of two discs and requires a minimum of four discswith a storage capacity of n−2.

While RAID addresses the memory device failure issue, it is not withoutits own failures issues that affect its effectiveness, efficiency andsecurity. For instance, as more discs are added to the array, theprobability of a disc failure increases, which increases the demand formaintenance. For example, when a disc fails, it needs to be manuallyreplaced before another disc fails and the data stored in the RAIDdevice is lost. To reduce the risk of data loss, data on a RAID deviceis typically copied on to one or more other RAID devices. While thisaddresses the loss of data issue, it raises a security issue sincemultiple copies of data are available, which increases the chances ofunauthorized access. Further, as the amount of data being stored grows,the overhead of RAID devices becomes a non-trivial efficiency issue.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

FIG. 1 is a schematic block diagram of an embodiment of a computingsystem in accordance with the present invention;

FIG. 2 is a schematic block diagram of an embodiment of a computing corein accordance with the present invention;

FIG. 3 is a schematic block diagram of an embodiment of a distributedstorage processing unit in accordance with the present invention;

FIG. 4 is a schematic block diagram of an embodiment of a grid module inaccordance with the present invention;

FIG. 5 is a diagram of an example embodiment of error coded data slicecreation in accordance with the present invention;

FIG. 6A is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 6B is a flowchart illustrating an example of storing key shares inaccordance with the present invention;

FIG. 6C is a diagram illustrating an example of a key share storagetable in accordance with the present invention;

FIG. 6D is a flowchart illustrating an example of generating a signaturein accordance with the present invention;

FIG. 6E is a flowchart illustrating an example of generating a signaturecontribution in accordance with the present invention;

FIG. 7 is a flowchart illustrating another example of generating asignature contribution in accordance with the present invention;

FIG. 8 is a flowchart illustrating an example of replicating encodeddata slices in accordance with the present invention;

FIG. 9A is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 9B is a timing diagram illustrating an example of pre-fetching adata segment in accordance with the present invention;

FIG. 9C is a flowchart illustrating an example of pre-fetching a datasegment in accordance with the present invention;

FIG. 10A is a diagram illustrating an example of encoding data inaccordance with the present invention;

FIG. 10B is a flowchart illustrating an example of encoding data inaccordance with the present invention;

FIG. 10C is a flowchart illustrating an example of decoding data inaccordance with the present invention;

FIG. 11A is a diagram illustrating an example of appending data inaccordance with the present invention;

FIG. 11B is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 11C is a flowchart illustrating an example of appending data inaccordance with the present invention;

FIG. 11D is a flowchart illustrating an example of appending sliceportions in accordance with the present invention;

FIG. 12A is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 12B is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 12C is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 12D is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 12E is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 13A is a flowchart illustrating an example of rebuilding a Shamirsecret share in accordance with the present invention;

FIG. 13B is a flowchart illustrating an example of generating a rebuiltshare partial in accordance with the present invention;

FIG. 14A is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 14B is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 14C is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 14D is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 15A is a schematic block diagram of an embodiment of a dispersedstorage system in accordance with the present invention;

FIG. 15B is a schematic block diagram of another embodiment of acomputing system in accordance with the present invention;

FIG. 15C is a flowchart illustrating an example of updating errorrecovery information in accordance with the present invention;

FIG. 16 is a diagram illustrating an example of a directory filestructure in accordance with the present invention;

FIG. 17 is a flowchart illustrating an example of deleting a snapshot inaccordance with the present invention;

FIG. 18 is a diagram illustrating another example of a directory filestructure in accordance with the present invention; and

FIG. 19 is a flowchart illustrating another example of deleting asnapshot in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a schematic block diagram of a computing system 10 thatincludes one or more of a first type of user devices 12, one or more ofa second type of user devices 14, at least one distributed storage (DS)processing unit 16, at least one DS managing unit 18, at least onestorage integrity processing unit 20, and a distributed storage network(DSN) memory 22 coupled via a network 24. The network 24 may include oneor more wireless and/or wire lined communication systems; one or moreprivate intranet systems and/or public internet systems; and/or one ormore local area networks (LAN) and/or wide area networks (WAN).

The DSN memory 22 includes a plurality of distributed storage (DS) units36 for storing data of the system. Each of the DS units 36 includes aprocessing module and memory and may be located at a geographicallydifferent site than the other DS units (e.g., one in Chicago, one inMilwaukee, etc.).

Each of the user devices 12-14, the DS processing unit 16, the DSmanaging unit 18, and the storage integrity processing unit 20 may be aportable computing device (e.g., a social networking device, a gamingdevice, a cell phone, a smart phone, a personal digital assistant, adigital music player, a digital video player, a laptop computer, ahandheld computer, a video game controller, and/or any other portabledevice that includes a computing core) and/or a fixed computing device(e.g., a personal computer, a computer server, a cable set-top box, asatellite receiver, a television set, a printer, a fax machine, homeentertainment equipment, a video game console, and/or any type of homeor office computing equipment). Such a portable or fixed computingdevice includes a computing core 26 and one or more interfaces 30, 32,and/or 33. An embodiment of the computing core 26 will be described withreference to FIG. 2.

With respect to the interfaces, each of the interfaces 30, 32, and 33includes software and/or hardware to support one or more communicationlinks via the network 24 indirectly and/or directly. For example,interfaces 30 support a communication link (wired, wireless, direct, viaa LAN, via the network 24, etc.) between the second type of user device14 and the DS processing unit 16. As another example, DSN interface 32supports a plurality of communication links via the network 24 betweenthe DSN memory 22 and the DS processing unit 16, the first type of userdevice 12, and/or the storage integrity processing unit 20. As yetanother example, interface 33 supports a communication link between theDS managing unit 18 and any one of the other devices and/or units 12,14, 16, 20, and/or 22 via the network 24.

In general and with respect to data storage, the system 10 supportsthree primary functions: distributed network data storage management,distributed data storage and retrieval, and data storage integrityverification. In accordance with these three primary functions, data canbe distributedly stored in a plurality of physically different locationsand subsequently retrieved in a reliable and secure manner regardless offailures of individual storage devices, failures of network equipment,the duration of storage, the amount of data being stored, attempts athacking the data, etc.

The DS managing unit 18 performs distributed network data storagemanagement functions, which include establishing distributed datastorage parameters, performing network operations, performing networkadministration, and/or performing network maintenance. The DS managingunit 18 establishes the distributed data storage parameters (e.g.,allocation of virtual DSN memory space, distributed storage parameters,security parameters, billing information, user profile information,etc.) for one or more of the user devices 12-14 (e.g., established forindividual devices, established for a user group of devices, establishedfor public access by the user devices, etc.). For example, the DSmanaging unit 18 coordinates the creation of a vault (e.g., a virtualmemory block) within the DSN memory 22 for a user device (for a group ofdevices, or for public access). The DS managing unit 18 also determinesthe distributed data storage parameters for the vault. In particular,the DS managing unit 18 determines a number of slices (e.g., the numberthat a data segment of a data file and/or data block is partitioned intofor distributed storage) and a read threshold value (e.g., the minimumnumber of slices required to reconstruct the data segment).

As another example, the DS managing module 18 creates and stores,locally or within the DSN memory 22, user profile information. The userprofile information includes one or more of authentication information,permissions, and/or the security parameters. The security parameters mayinclude one or more of encryption/decryption scheme, one or moreencryption keys, key generation scheme, and data encoding/decodingscheme.

As yet another example, the DS managing unit 18 creates billinginformation for a particular user, user group, vault access, publicvault access, etc. For instance, the DS managing unit 18 tracks thenumber of times user accesses a private vault and/or public vaults,which can be used to generate a per-access bill. In another instance,the DS managing unit 18 tracks the amount of data stored and/orretrieved by a user device and/or a user group, which can be used togenerate a per-data-amount bill.

The DS managing unit 18 also performs network operations, networkadministration, and/or network maintenance. As at least part ofperforming the network operations and/or administration, the DS managingunit 18 monitors performance of the devices and/or units of the system10 for potential failures, determines the devices and/or unit'sactivation status, determines the devices' and/or units' loading, andany other system level operation that affects the performance level ofthe system 10. For example, the DS managing unit 18 receives andaggregates network management alarms, alerts, errors, statusinformation, performance information, and messages from the devices12-14 and/or the units 16, 20, 22. For example, the DS managing unit 18receives a simple network management protocol (SNMP) message regardingthe status of the DS processing unit 16.

The DS managing unit 18 performs the network maintenance by identifyingequipment within the system 10 that needs replacing, upgrading,repairing, and/or expanding. For example, the DS managing unit 18determines that the DSN memory 22 needs more DS units 36 or that one ormore of the DS units 36 needs updating.

The second primary function (i.e., distributed data storage andretrieval) begins and ends with a user device 12-14. For instance, if asecond type of user device 14 has a data file 38 and/or data block 40 tostore in the DSN memory 22, it send the data file 38 and/or data block40 to the DS processing unit 16 via its interface 30. As will bedescribed in greater detail with reference to FIG. 2, the interface 30functions to mimic a conventional operating system (OS) file systeminterface (e.g., network file system (NFS), flash file system (FFS),disk file system (DFS), file transfer protocol (FTP), web-baseddistributed authoring and versioning (WebDAV), etc.) and/or a blockmemory interface (e.g., small computer system interface (SCSI), internetsmall computer system interface (iSCSI), etc.). In addition, theinterface 30 may attach a user identification code (ID) to the data file38 and/or data block 40.

The DS processing unit 16 receives the data file 38 and/or data block 40via its interface 30 and performs a distributed storage (DS) process 34thereon (e.g., an error coding dispersal storage function). The DSprocessing 34 begins by partitioning the data file 38 and/or data block40 into one or more data segments, which is represented as Y datasegments. For example, the DS processing 34 may partition the data file38 and/or data block 40 into a fixed byte size segment (e.g., 2¹ to2^(n) bytes, where n=>2) or a variable byte size (e.g., change byte sizefrom segment to segment, or from groups of segments to groups ofsegments, etc.).

For each of the Y data segments, the DS processing 34 error encodes(e.g., forward error correction (FEC), information dispersal algorithm,or error correction coding) and slices (or slices then error encodes)the data segment into a plurality of error coded (EC) data slices 42-48,which is represented as X slices per data segment. The number of slices(X) per segment, which corresponds to a number of pillars n, is set inaccordance with the distributed data storage parameters and the errorcoding scheme. For example, if a Reed-Solomon (or other FEC scheme) isused in an n/k system, then a data segment is divided into n slices,where k number of slices is needed to reconstruct the original data(i.e., k is the threshold). As a few specific examples, the n/k factormay be 5/3; 6/4; 8/6; 8/5; 16/10.

For each slice 42-48, the DS processing unit 16 creates a unique slicename and appends it to the corresponding slice 42-48. The slice nameincludes universal DSN memory addressing routing information (e.g.,virtual memory addresses in the DSN memory 22) and user-specificinformation (e.g., user ID, file name, data block identifier, etc.).

The DS processing unit 16 transmits the plurality of EC slices 42-48 toa plurality of DS units 36 of the DSN memory 22 via the DSN interface 32and the network 24. The DSN interface 32 formats each of the slices fortransmission via the network 24. For example, the DSN interface 32 mayutilize an internet protocol (e.g., TCP/IP, etc.) to packetize theslices 42-48 for transmission via the network 24.

The number of DS units 36 receiving the slices 42-48 is dependent on thedistributed data storage parameters established by the DS managing unit18. For example, the DS managing unit 18 may indicate that each slice isto be stored in a different DS unit 36. As another example, the DSmanaging unit 18 may indicate that like slice numbers of different datasegments are to be stored in the same DS unit 36. For example, the firstslice of each of the data segments is to be stored in a first DS unit36, the second slice of each of the data segments is to be stored in asecond DS unit 36, etc. In this manner, the data is encoded anddistributedly stored at physically diverse locations to improved datastorage integrity and security.

Each DS unit 36 that receives a slice 42-48 for storage translates thevirtual DSN memory address of the slice into a local physical addressfor storage. Accordingly, each DS unit 36 maintains a virtual tophysical memory mapping to assist in the storage and retrieval of data.

The first type of user device 12 performs a similar function to storedata in the DSN memory 22 with the exception that it includes the DSprocessing. As such, the device 12 encodes and slices the data fileand/or data block it has to store. The device then transmits the slices11 to the DSN memory via its DSN interface 32 and the network 24.

For a second type of user device 14 to retrieve a data file or datablock from memory, it issues a read command via its interface 30 to theDS processing unit 16. The DS processing unit 16 performs the DSprocessing 34 to identify the DS units 36 storing the slices of the datafile and/or data block based on the read command. The DS processing unit16 may also communicate with the DS managing unit 18 to verify that theuser device 14 is authorized to access the requested data.

Assuming that the user device is authorized to access the requesteddata, the DS processing unit 16 issues slice read commands to at least athreshold number of the DS units 36 storing the requested data (e.g., toat least 10 DS units for a 16/10 error coding scheme). Each of the DSunits 36 receiving the slice read command, verifies the command,accesses its virtual to physical memory mapping, retrieves the requestedslice, or slices, and transmits it to the DS processing unit 16.

Once the DS processing unit 16 has received a read threshold number ofslices for a data segment, it performs an error decoding function andde-slicing to reconstruct the data segment. When Y number of datasegments has been reconstructed, the DS processing unit 16 provides thedata file 38 and/or data block 40 to the user device 14. Note that thefirst type of user device 12 performs a similar process to retrieve adata file and/or data block.

The storage integrity processing unit 20 performs the third primaryfunction of data storage integrity verification. In general, the storageintegrity processing unit 20 periodically retrieves slices 45, and/orslice names, of a data file or data block of a user device to verifythat one or more slices have not been corrupted or lost (e.g., the DSunit failed). The retrieval process mimics the read process previouslydescribed.

If the storage integrity processing unit 20 determines that one or moreslices is corrupted or lost, it rebuilds the corrupted or lost slice(s)in accordance with the error coding scheme. The storage integrityprocessing unit 20 stores the rebuild slice, or slices, in theappropriate DS unit(s) 36 in a manner that mimics the write processpreviously described.

FIG. 2 is a schematic block diagram of an embodiment of a computing core26 that includes a processing module 50, a memory controller 52, mainmemory 54, a video graphics processing unit 55, an input/output (IO)controller 56, a peripheral component interconnect (PCI) interface 58,at least one IO device interface module 62, a read only memory (ROM)basic input output system (BIOS) 64, and one or more memory interfacemodules. The memory interface module(s) includes one or more of auniversal serial bus (USB) interface module 66, a host bus adapter (HBA)interface module 68, a network interface module 70, a flash interfacemodule 72, a hard drive interface module 74, and a DSN interface module76. Note the DSN interface module 76 and/or the network interface module70 may function as the interface 30 of the user device 14 of FIG. 1.Further note that the IO device interface module 62 and/or the memoryinterface modules may be collectively or individually referred to as IOports.

FIG. 3 is a schematic block diagram of an embodiment of a dispersedstorage (DS) processing module 34 of user device 12 and/or of the DSprocessing unit 16. The DS processing module 34 includes a gatewaymodule 78, an access module 80, a grid module 82, and a storage module84. The DS processing module 34 may also include an interface 30 and theDSnet interface 32 or the interfaces 68 and/or 70 may be part of user 12or of the DS processing unit 14. The DS processing module 34 may furtherinclude a bypass/feedback path between the storage module 84 to thegateway module 78. Note that the modules 78-84 of the DS processingmodule 34 may be in a single unit or distributed across multiple units.

In an example of storing data, the gateway module 78 receives anincoming data object that includes a user ID field 86, an object namefield 88, and the data field 40 and may also receive correspondinginformation that includes a process identifier (e.g., an internalprocess/application ID), metadata, a file system directory, a blocknumber, a transaction message, a user device identity (ID), a dataobject identifier, a source name, and/or user information. The gatewaymodule 78 authenticates the user associated with the data object byverifying the user ID 86 with the managing unit 18 and/or anotherauthenticating unit.

When the user is authenticated, the gateway module 78 obtains userinformation from the management unit 18, the user device, and/or theother authenticating unit. The user information includes a vaultidentifier, operational parameters, and user attributes (e.g., userdata, billing information, etc.). A vault identifier identifies a vault,which is a virtual memory space that maps to a set of DS storage units36. For example, vault 1 (i.e., user 1's DSN memory space) includeseight DS storage units (X=8 wide) and vault 2 (i.e., user 2's DSN memoryspace) includes sixteen DS storage units (X=16 wide). The operationalparameters may include an error coding algorithm, the width n (number ofpillars X or slices per segment for this vault), a read threshold T, awrite threshold, an encryption algorithm, a slicing parameter, acompression algorithm, an integrity check method, caching settings,parallelism settings, and/or other parameters that may be used to accessthe DSN memory layer.

The gateway module 78 uses the user information to assign a source name35 to the data. For instance, the gateway module 78 determines thesource name 35 of the data object 40 based on the vault identifier andthe data object. For example, the source name may contain a fileidentifier (ID), a vault generation number, a reserved field, and avault identifier (ID). As another example, the gateway module 78 maygenerate the file ID based on a hash function of the data object 40.Note that the gateway module 78 may also perform message conversion,protocol conversion, electrical conversion, optical conversion, accesscontrol, user identification, user information retrieval, trafficmonitoring, statistics generation, configuration, management, and/orsource name determination.

The access module 80 receives the data object 40 and creates a series ofdata segments 1 through Y 90-92 in accordance with a data storageprotocol (e.g., file storage system, a block storage system, and/or anaggregated block storage system). The number of segments Y may be chosenor randomly assigned based on a selected segment size and the size ofthe data object. For example, if the number of segments is chosen to bea fixed number, then the size of the segments varies as a function ofthe size of the data object. For instance, if the data object is animage file of 4,194,304 eight bit bytes (e.g., 33,554,432 bits) and thenumber of segments Y=131,072, then each segment is 256 bits or 32 bytes.As another example, if segment sized is fixed, then the number ofsegments Y varies based on the size of data object. For instance, if thedata object is an image file of 4,194,304 bytes and the fixed size ofeach segment is 4,096 bytes, the then number of segments Y=1,024. Notethat each segment is associated with the same source name.

The grid module 82 receives the data segments and may manipulate (e.g.,compression, encryption, cyclic redundancy check (CRC), etc.) each ofthe data segments before performing an error coding function of theerror coding dispersal storage function to produce a pre-manipulateddata segment. After manipulating a data segment, if applicable, the gridmodule 82 error encodes (e.g., Reed-Solomon, Convolution encoding,Trellis encoding, etc.) the data segment or manipulated data segmentinto X error coded data slices 42-44.

The value X, or the number of pillars (e.g., X=16), is chosen as aparameter of the error coding dispersal storage function. Otherparameters of the error coding dispersal function include a readthreshold T, a write threshold W, etc. The read threshold (e.g., T=10,when X=16) corresponds to the minimum number of error-free error codeddata slices required to reconstruct the data segment. In other words,the DS processing module 34 can compensate for X-T (e.g., 16−10=6)missing error coded data slices per data segment. The write threshold Wcorresponds to a minimum number of DS storage units that acknowledgeproper storage of their respective data slices before the DS processingmodule indicates proper storage of the encoded data segment. Note thatthe write threshold is greater than or equal to the read threshold for agiven number of pillars (X).

For each data slice of a data segment, the grid module 82 generates aunique slice name 37 and attaches it thereto. The slice name 37 includesa universal routing information field and a vault specific field and maybe 48 bytes (e.g., 24 bytes for each of the universal routinginformation field and the vault specific field). As illustrated, theuniversal routing information field includes a slice index, a vault ID,a vault generation, and a reserved field. The slice index is based onthe pillar number and the vault ID and, as such, is unique for eachpillar (e.g., slices of the same pillar for the same vault for anysegment will share the same slice index). The vault specific fieldincludes a data name, which includes a file ID and a segment number(e.g., a sequential numbering of data segments 1−Y of a simple dataobject or a data block number).

Prior to outputting the error coded data slices of a data segment, thegrid module may perform post-slice manipulation on the slices. Ifenabled, the manipulation includes slice level compression, encryption,CRC, addressing, tagging, and/or other manipulation to improve theeffectiveness of the computing system.

When the error coded data slices of a data segment are ready to beoutputted, the grid module 82 determines which of the DS storage units36 will store the EC data slices based on a dispersed storage memorymapping associated with the user's vault and/or DS storage unitattributes. The DS storage unit attributes may include availability,self-selection, performance history, link speed, link latency,ownership, available DSN memory, domain, cost, a prioritization scheme,a centralized selection message from another source, a lookup table,data ownership, and/or any other factor to optimize the operation of thecomputing system. Note that the number of DS storage units 36 is equalto or greater than the number of pillars (e.g., X) so that no more thanone error coded data slice of the same data segment is stored on thesame DS storage unit 36. Further note that EC data slices of the samepillar number but of different segments (e.g., EC data slice 1 of datasegment 1 and EC data slice 1 of data segment 2) may be stored on thesame or different DS storage units 36.

The storage module 84 performs an integrity check on the outboundencoded data slices and, when successful, identifies a plurality of DSstorage units based on information provided by the grid module 82. Thestorage module 84 then outputs the encoded data slices 1 through X ofeach segment 1 through Y to the DS storage units 36. Each of the DSstorage units 36 stores its EC data slice(s) and maintains a localvirtual DSN address to physical location table to convert the virtualDSN address of the EC data slice(s) into physical storage addresses.

In an example of a read operation, the user device 12 and/or 14 sends aread request to the DS processing unit 14, which authenticates therequest. When the request is authentic, the DS processing unit 14 sendsa read message to each of the DS storage units 36 storing slices of thedata object being read. The slices are received via the DSnet interface32 and processed by the storage module 84, which performs a parity checkand provides the slices to the grid module 82 when the parity check wassuccessful. The grid module 82 decodes the slices in accordance with theerror coding dispersal storage function to reconstruct the data segment.The access module 80 reconstructs the data object from the data segmentsand the gateway module 78 formats the data object for transmission tothe user device.

FIG. 4 is a schematic block diagram of an embodiment of a grid module 82that includes a control unit 73, a pre-slice manipulator 75, an encoder77, a slicer 79, a post-slice manipulator 81, a pre-slice de-manipulator83, a decoder 85, a de-slicer 87, and/or a post-slice de-manipulator 89.Note that the control unit 73 may be partially or completely external tothe grid module 82. For example, the control unit 73 may be part of thecomputing core at a remote location, part of a user device, part of theDS managing unit 18, or distributed amongst one or more DS storageunits.

In an example of write operation, the pre-slice manipulator 75 receivesa data segment 90-92 and a write instruction from an authorized userdevice. The pre-slice manipulator 75 determines if pre-manipulation ofthe data segment 90-92 is required and, if so, what type. The pre-slicemanipulator 75 may make the determination independently or based oninstructions from the control unit 73, where the determination is basedon a computing system-wide predetermination, a table lookup, vaultparameters associated with the user identification, the type of data,security requirements, available DSN memory, performance requirements,and/or other metadata.

Once a positive determination is made, the pre-slice manipulator 75manipulates the data segment 90-92 in accordance with the type ofmanipulation. For example, the type of manipulation may be compression(e.g., Lempel-Ziv-Welch, Huffman, Golomb, fractal, wavelet, etc.),signatures (e.g., Digital Signature Algorithm (DSA), Elliptic Curve DSA,Secure Hash Algorithm, etc.), watermarking, tagging, encryption (e.g.,Data Encryption Standard, Advanced Encryption Standard, etc.), addingmetadata (e.g., time/date stamping, user information, file type, etc.),cyclic redundancy check (e.g., CRC32), and/or other data manipulationsto produce the pre-manipulated data segment.

The encoder 77 encodes the pre-manipulated data segment 92 using aforward error correction (FEC) encoder (and/or other type of erasurecoding and/or error coding) to produce an encoded data segment 94. Theencoder 77 determines which forward error correction algorithm to usebased on a predetermination associated with the user's vault, a timebased algorithm, user direction, DS managing unit direction, controlunit direction, as a function of the data type, as a function of thedata segment 92 metadata, and/or any other factor to determine algorithmtype. The forward error correction algorithm may be Golay,Multidimensional parity, Reed-Solomon, Hamming, Bose Ray ChauduriHocquenghem (BCH), Cauchy-Reed-Solomon, or any other FEC encoder. Notethat the encoder 77 may use a different encoding algorithm for each datasegment 92, the same encoding algorithm for the data segments 92 of adata object, or a combination thereof.

The encoded data segment 94 is of greater size than the data segment 92by the overhead rate of the encoding algorithm by a factor of X/T, whereX is the width or number of slices, and T is the read threshold. In thisregard, the corresponding decoding process can accommodate at most X-Tmissing EC data slices and still recreate the data segment 92. Forexample, if X=16 and T=10, then the data segment 92 will be recoverableas long as 10 or more EC data slices per segment are not corrupted.

The slicer 79 transforms the encoded data segment 94 into EC data slicesin accordance with the slicing parameter from the vault for this userand/or data segment 92. For example, if the slicing parameter is X=16,then the slicer 79 slices each encoded data segment 94 into 16 encodedslices.

The post-slice manipulator 81 performs, if enabled, post-manipulation onthe encoded slices to produce the EC data slices. If enabled, thepost-slice manipulator 81 determines the type of post-manipulation,which may be based on a computing system-wide predetermination,parameters in the vault for this user, a table lookup, the useridentification, the type of data, security requirements, available DSNmemory, performance requirements, control unit directed, and/or othermetadata. Note that the type of post-slice manipulation may includeslice level compression, signatures, encryption, CRC, addressing,watermarking, tagging, adding metadata, and/or other manipulation toimprove the effectiveness of the computing system.

In an example of a read operation, the post-slice de-manipulator 89receives at least a read threshold number of EC data slices and performsthe inverse function of the post-slice manipulator 81 to produce aplurality of encoded slices. The de-slicer 87 de-slices the encodedslices to produce an encoded data segment 94. The decoder 85 performsthe inverse function of the encoder 77 to recapture the data segment90-92. The pre-slice de-manipulator 83 performs the inverse function ofthe pre-slice manipulator 75 to recapture the data segment 90-92.

FIG. 5 is a diagram of an example of slicing an encoded data segment 94by the slicer 79. In this example, the encoded data segment 94 includesthirty-two bits, but may include more or less bits. The slicer 79disperses the bits of the encoded data segment 94 across the EC dataslices in a pattern as shown. As such, each EC data slice does notinclude consecutive bits of the data segment 94 reducing the impact ofconsecutive bit failures on data recovery. For example, if EC data slice2 (which includes bits 1, 5, 9, 13, 17, 25, and 29) is unavailable(e.g., lost, inaccessible, or corrupted), the data segment can bereconstructed from the other EC data slices (e.g., 1, 3 and 4 for a readthreshold of 3 and a width of 4).

FIG. 6A is a schematic block diagram of another embodiment of acomputing system that includes a device 102 (e.g., a user device 12-14,a dispersed storage (DS) processing unit 16, a DS managing unit 18, a DSunit 36, and a storage integrity processing unit 20) and a dispersedstorage network (DSN) memory 22. The DSN memory 22 includes a pluralityof DS units 36. The device 102 includes a module 104 to enable thedevice 102 to generate a secure signature on an item without a locallystored private key 116 of the device 102. The private key 116 may beobtained by at least one of receiving, generating a private/public keypair, generating based on an attribute associated with the device 102,and generating based on a random number to produce a random key asprivate key 116. The item includes a data element that includes one ormore of registry information, key information, encryption algorithminformation, a device certificate, a user certificate, a system elementidentifier (ID), a DSN access request, and a hash of the data element,wherein a hashing function is applied to the data element to produce thehash of the data element. The module 104 includes a select key indexmodule 106, an obtain signature contributions module 108, a generatesignature module 110, a populate key shares module 112, and a validatesignature module 114.

The select key index module 106 is operable to select a first keyrepresentation index 118 of a set of key representation indexes, whereinthe first key representation index 118 includes information regarding afirst key representation of a set of key representations, wherein afirst mathematical encoding of the private key 116 generates a firstplurality of key shares 122 as the first key representation, which isstored in a first set of DS units 36 of the DSN memory 22, and a secondmathematical encoding of the private key 116 generates a secondplurality of key shares 124 as a second key representation of the set ofkey representations, which is stored in a second set of DS units 36 ofthe DSN memory 22. The select key index module 106 functions to selectthe first key representation index 118 by selecting the first keyrepresentation index 118 based on one or more of DS unit statusindicators of the first and second sets of DS units, DS unit performancelevel indicators of the first and second sets of DS units, DS unitretrieval history indicators of the first and second sets of DS units, asecurity indicator, a query, a command, a key share storage tablelookup, and a message. For example, the select key index module 106selects the first key representation index 118 when DS unit performancelevel indicators associated with the first set of DS units 36 arefavored (e.g., lower average retrieval access latency time) over DS unitperformance level indicators associated with the second set of DS units36.

The first mathematical encoding includes the populate key shares module112 operable to generate at least one set of key shares for storage inone or more sets of DS units, the generating includes the populate keyshares module 112 of operable to generate one or more first values,generate a second value based on the one or more first values, theprivate key 116, and a key share generating mathematical function, andsend the one or more first values and the second value to the first setof DS units. The second mathematical encoding includes the populate keyshares module 112 further operable to generate one or more third values,generate a fourth value based on the one or more third values, theprivate key 116, and the key share generating mathematical function, andsend the one or more third values and the fourth value to the second setof DS units. The populate key shares module 112 is further operable torandomly generate the one or more first values and generate the secondvalue based on key share generating mathematical function of (x+y+z) modΦ (n)=d, where d is the private key 116, n is a public modulus (e.g., a1024 bit wide public key paired to the private key 116), x and ycorrespond to the one or more first values, z corresponds to the secondvalue, φ (n) is an Euler's totient function φ (n)=(p−1)*(q−1), and p andq are large prime numbers in accordance with n=p*q. After generating theset of key representations, the populate key shares module 112 destroysthe private key 116.

The obtain signature contributions module 108 is operable to determinewhether a first plurality of signature contributions 126 have beenreceived in response to a signature request 128 for the item based onthe first key representation index 118, wherein one of the first set ofDS units executes a first mathematical signature function using one ofthe first plurality of key shares on the item to produce a signaturecontribution of the first plurality of signature contributions. Forexample, the one of the first set of DS units executes the firstmathematical signature function as first signaturecontribution=(item)^(keyshare) mod n, wherein the item is extracted fromthe request 128, the keyshare is retrieved as a corresponding value froma local memory of the one DS unit, and n is a public modulus extractedfrom the signature request 128. The signature request 128 includes a setof signature requests, wherein each signature request of the set ofsignature requests includes one or more of the first key index 118(e.g., a share set number, a DS unit combination ID, DS unit IDs of thecorresponding DS units), a signature payload (e.g., the item to sign), ahash of the signature payload, a DS unit ID, and a public modulus value(e.g., n).

When the first plurality of signature contributions 126 have beenreceived, the generate signature module 110 is operable to generate thesecure signature 130 on the item from the first plurality of signaturecontributions 126. The generate signature module 110 functions togenerate the secure signature 130 on the item by multiplying the firstplurality of signature contributions 126 to produce a multiplicationresult and performing a modulus function on the multiplication result toproduce the secure signature 130. For example, to generate signaturemodule 110 produces the secure signature 130 on the item in accordancewith (contribution 1*contribution 2*contribution 3) mod n, when thefirst plurality of signature contributions 126 includes three signaturecontributions.

When the first plurality of signature contributions have not beenreceived (e.g., receiving two when the plurality includes three; notreceiving all three within a time period), the obtain signaturecontributions module 108 is further operable to generate secondsignature requests 132 for the item based on a second key representationindex 120 of the set of key representation indexes, send the secondsignature requests 132 to the second set of DS units, determine whethera second plurality of signature contributions 134 have been receivedfrom the second set of DS units, and when the second plurality ofsignature contributions 134 have been received, the generate signaturemodule 110 is further operable to generate the secure signature 130 onthe item from the second plurality of signature contributions. Forexample, the select key index module 106 identifies the second keyrepresentation index 120 when DS unit performance level indicatorsassociated with the second set of DS units are more favorable (e.g.,lower average retrieval access latency time) than DS unit performancelevel indicators associated with other sets of DS units.

The validate signature module 114 is operable to determine whether thesecure signature is valid. For example, the validate signature module114 indicates that the secure signature 130 is not valid when a hash ofthe item is not substantially the same as a decrypted secure signature,wherein the secure signature 130 is decrypted utilizing a public keyassociated with the private key 116. When the secure signature 130 isnot valid, the obtain signature contributions module 108 is furtheroperable to generate second signature requests 132 for the item based onthe second key representation index 120 of the set of key representationindexes, send the second signature requests 132 to the second set of DSunits, determine whether the second plurality of signature contributions134 have been received from the second set of DS units. When the secondplurality of signature contributions 134 have been received, thegenerate signature module 110 is further operable to generate the securesignature 130 on the item from the second plurality of signaturecontributions 134. The validate signature module 114 is further operableto determine whether the secure signature 130 on the item from thesecond plurality of signature contributions 134 is valid.

FIG. 6B is a flowchart illustrating an example of storing key shares.The method begins at step 144 where a processing module obtains aprivate key (e.g., an encryption key d utilized in an encryptionalgorithm). The obtaining may be based on one or more of generating aprivate/public key pair (e.g., a private key and a key), receiving theprivate key, a query, a lookup, and a user input. The method continuesat step 146 where the processing module determines sharing functionparameters. The sharing function parameters includes one or more of awidth number (e.g., a number of storage nodes w), a decode threshold(e.g., a number of key shares k), a number of shares sets (e.g., wchoose k), a public modulus n (e.g., a public key), security functionconstants p and q (e.g., large primes such that p*q=n), an encryptionalgorithm identifier (ID), and a decryption algorithm ID. Thedetermining may be based on one or more of a list, a predetermination, aquery, a performance level indicator, a reliability level requirement, amessage, and a command. For example, the processing module determinesw=4, k=2, a number of shares sets=(4 choose 2)=6, a value for n, and avalue for p based on a lookup and generates a value for q in accordancewith p*q=n.

The method continues at step 148 where the processing module identifiesa plurality of storage nodes for storing key shares. The plurality ofstorage nodes includes two or more of a user device, a dispersed storage(DS) unit, a storage server, and a memory device. The navigation may bebased on one or more of the sharing function parameters, a list, apredetermination, a query, a performance level indicator, a message, anda command. For example, the processing module identifies the pluralityof storage nodes to include 4 DS units when the sharing functionparameters include a width w=4 and a performance level indicatorindicates that a performance level of the 4 DS units compares favorablyto a performance level threshold.

The method continues at step 150 where the processing module generatesone or more sets of key shares to include the number of shares sets. Forexample, the processing module generates 6 sets of key shares when thewidth w=4 and the decode threshold k=2 (e.g., 4 choose 2=6). Thegeneration produces a set of key shares for each combination of a decodethreshold k number of key shares stored in the width w number of storagenodes. The generation of each set of key shares includes generation inaccordance with a formula (x+y+z) mod Φ (n)=private key d, wherein φ(n)=(p−1)* (q−1), n=p*q, and x, y, z represent key shares of acorresponding key share set when a number of key shares is three. Forexample, the processing module randomly chooses values for key shares yand z of a corresponding key share set and generates a value for keyshare x in accordance with the formula.

In an example of generating key share sets, the processing modulegenerates 10 key shares sets to include a first key share set thatincludes a key share x1 to store in DS unit 1, a key share y1 to storein DS unit 2, and a key share z1 to store in DS unit 3, a second keyshare set that includes a key share x2 to store in DS unit 1, a keyshare y2 to store in DS unit 2, and a key share z2 to store in DS unit4, a third key share set that includes a key share x3 to store in DSunit 1, a key share y3 to store in DS unit 2, and a key share z3 tostore in DS unit 5, a fourth key share set that includes a key share x4to store in DS unit 1, a key share y4 to store in DS unit 3, and a keyshare z4 to store in DS unit 5, a fifth key share set that includes akey share x5 to store in DS unit 1, a key share y5 to store in DS unit4, and a key share z5 to store in DS unit 5, a sixth key share set thatincludes a key share x6 to store in DS unit 2, a key share y6 to storein DS unit 3, and a key share z6 to store in DS unit 4, a seventh keyshare set that includes a key share x7 to store in DS unit 2, a keyshare y7 to store in DS unit 3, and a key share z7 to store in DS unit5, an eighth key share set that includes a key share x8 to store in DSunit 2, a key share y7 to store in DS unit 4, and a key share z7 tostore in DS unit 5, a ninth key share set that includes a key share x9to store in DS unit 3, a key share y9 to store in DS unit 4, and a keyshare z9 to store in DS unit 5, and a 10th key share set that includes akey share x10 to store in DS unit 1, a key share y10 to store in DS unit3, and a key share z10 to store in DS unit 4 when a number storage nodesis 5 and a decode threshold is 3.

The method continues at step 152 where the processing module outputs theone or more sets of key shares to the plurality of storage nodes. Inaddition, the processing module may output one or more of the sharingfunction parameters to each storage node of the plurality of storagenodes. For example, the processing module sends the public modulus n toeach storage node of the plurality of storage nodes. The methodcontinues at the step where the processing module destroys the privatekey d. The destroying of the private key may provide the system with asecurity performance improvement. A method to generate a signature basedon stored shared keys is described in greater detail with reference toFIG. 6D. A method to generate a signature contribution (e.g., by astorage node) is described in greater detail with reference to FIG. 6E.

FIG. 6C is a diagram illustrating an example of a key share storagetable 156 that includes a share set field 158, a node combination field160, and a key share per storage node field 162. The share set field 158includes a share sets number of share set identifiers of correspondingkey share sets. For example, the share set field 158 includes 6 shareset identifiers 1-6 when a number of storage nodes w=4, a decodethreshold k=2, and the share sets number of share set identifiers is wchoose k (e.g., 4 choose 2=6). The node combination field 160 includes ashare sets number of node combination entries, wherein each nodecombination entry corresponds to a combination of a decode thresholdnumber of storage node identifiers. For example, the node combinationfield includes 6 node combination entries including A-B, A-C, A-D, B-C,B-D, and C-D when storage nodes A-D are utilized to store a share setsnumber (e.g., 6) of a decode threshold number (e.g., 2) of key shares.

The key share per storage node field 162 includes a share sets number ofstorage node fields, wherein each storage node field corresponds to astorage node of a number of storage nodes w utilized to store the keyshares. Each storage node field includes a key share identifier (e.g., akey representation index) utilized to identify a key share of anassociated share set (e.g., key representation) that is stored in astorage node corresponding to the storage node field. For example, shareset 1 includes utilization of storage node combination A-B such that keyshare x1 is stored in storage unit A and key share y1 is stored instorage node B, share set 2 includes utilization of storage nodecombination A-C such that key share x2 is stored in storage unit A andkey share y2 is stored in storage node C, share set 3 includesutilization of storage node combination A-D such that key share x3 isstored in storage unit A and key share y3 is stored in storage node D,share set 4 includes utilization of storage node combination B-C suchthat key share x4 is stored in storage unit B and key share y4 is storedin storage node C, share set 5 includes utilization of storage nodecombination B-D such that key share x5 is stored in storage unit B andkey share y5 is stored in storage node D, and share set 6 includesutilization of storage node combination C-D such that key share x6 isstored in storage unit C and key share y6 is stored in storage node D.Each set of key shares may be generated in accordance with a formula(x+y) mod Φ (n)=private key d, wherein Φ (n)=(p−1)*(q−1). For example, avalue of key share y1 is chosen randomly and a value for key share x1 isgenerated in accordance with the formula. Storing more than one keyshare set corresponding to more than one key representation of a privatekey in more than one storage node set may provide the system with anavailability improvement.

FIG. 6D is a flowchart illustrating an example of generating asignature, where a device of a distributed storage network (DSN)generates a secure signature on an item without a locally stored privatekey of the device. The item includes one or more of a data element thatincludes one or more of registry information, key information,encryption algorithm information, a device certificate, a usercertificate, and a system element identifier, and a DSN access requestand a hash of the data element.

The method begins at step 170 where a processing module selects a firstkey representation index of a set of key representation indexes, whereinthe first key representation index includes information regarding afirst key representation of a set of key representations, wherein afirst mathematical encoding of the private key generates a firstplurality of key shares as the first key representation, which is storedin a first set of dispersed storage (DS) units of the DSN, and a secondmathematical encoding of the private key generates a second plurality ofkey shares as a second key representation of the set of keyrepresentations, which is stored in a second set of dispersed storage(DS) units of the DSN. The first mathematical encoding includesgenerating one or more first values, generating a second value based onthe one or more first values, the private key, and a key sharegenerating mathematical function, and sending the one or more firstvalues and the second value to the first set of DS units. For example,the processing module randomly generates the one or more first valuesand generates the second value based on key share generatingmathematical function of (x+y+z) mod Φ (n)=d, where d is the privatekey, x and y correspond to the one or more first values, z correspondsto the second value, and Φ (n) is an Euler's totient function. Thesecond mathematical encoding includes generating one or more thirdvalues, generating a fourth value based on the one or more third values,the private key, and the key share generating mathematical function, andsending the one or more third values and the fourth value to the secondset of DS units. After generating the set of key representations, theprocessing module destroys the private key.

At step 170, the processing module selects the first key representationindex based on one or more of DS unit status indicators of the first andsecond sets of DS units, DS unit performance level indicators of thefirst and second sets of DS units, DS unit retrieval history indicatorsof the first and second sets of DS units, a security indicator, a query,a command, a key share storage table lookup, and a message. The methodcontinues at step 172 where the processing module determines whether afirst plurality of signature contributions have been received inresponse to a signature request for the item based on the first keyrepresentation index, wherein one of the first set of DS units executesa first mathematical signature function using one of the first pluralityof key shares on the item to produce a signature contribution of thefirst plurality of signature contributions. The signature requestincludes a set of signature requests, wherein each signature request ofthe set of signature requests includes one or more of a first keyrepresentation identifier (ID) (e.g., a share set number, a DS unitcombination ID, DS unit IDs of the corresponding DS units), a signaturepayload (e.g., something to sign like a certificate), a hash of thesignature payload, a DS unit ID, and a public modulus value (e.g., n).

The method branches to step 176 when the processing module determinesthat the first plurality of signature contributions have not beenreceived. The method continues to step 174 when the processing moduledetermines that the first plurality of signature contributions has beenreceived. The method continues at step 174 where the processing modulegenerates the secure signature on the item from the first plurality ofsignature contributions. The generating the secure signature on the itemincludes multiplying the first plurality of signature contributions toproduce a multiplication result and performing a modulus function on themultiplication result to produce the secure signature. The methodbranches to step 184.

The method continues at step 176 where the processing module generatessecond signature requests for the item based on a second keyrepresentation index of the set of key representation indexes when thefirst plurality of signature contributions have not been received. Themethod continues at step 178 where the processing module sends thesecond signature requests to the second set of DS units. The methodcontinues at step 180 where the processing module determines whether asecond plurality of signature contributions have been received from thesecond set of DS units. The method loops back to step 176 to sendanother signature request when the processing module determines that thesecond plurality of signature contributions have not been received(e.g., within a time period). The method continues to step 182 when theprocessing module determines that the second plurality of signaturecontributions have been received. The method continues at step 182 wherethe processing module generates the secure signature on the item fromthe second plurality of signature contributions.

The method continues at step 184 where the processing module determineswhether the secure signature is valid. For example, the processingmodule indicates that the secure signature is not valid when a hash ofthe item is not substantially the same as a decrypted secure signatureutilizing and associated public key. The method branches back to step176 to send another signature request when the processing moduledetermines that the secure signature is not valid. The method continuesto step 186 when the processing module determines that the securesignature is valid. The method continues at step 186 where theprocessing module indicates that the secure signature is valid. Theprocessing module may output the secure signature to a requestingentity.

FIG. 6E is a flowchart illustrating an example of generating a signaturecontribution. The method begins at step 190 where a processing module(e.g., of a storage node, of a dispersed storage (DS) unit) receives asignature contribution request. The method continues at step 192 wherethe processing module retrieves a key share based one or more of onsharing function parameters and the signature contribution request. Theprocessing module obtains the sharing function parameters based on oneor more of receiving the sharing function parameters, extracting thesharing function parameters from the signature contribution request, alookup, a message, a command, and a predetermination. For example, theprocessing module extracts the sharing function parameters from thesignature contribution request to include a key share identifier (ID).The retrieving of the key share includes obtaining the key share ID andretrieving the key share based on the key share ID. For example, theprocessing module obtains key share ID x3, determines a storage node Amemory location based on the key share ID x3, and retrieves the keyshare x3 from a memory of the storage node at the storage node memorylocation.

As another example, the processing module extracts the sharing functionparameters from the signature contribution request to include a shareset ID. The retrieving of the key share includes obtaining the share setID and retrieving the key share based on the share set ID and a keyshare storage table lookup. For example, the processing module obtainsshare set ID A-D, determines a key share ID of x3 based on a key sharestorage table lookup utilizing the share set ID A-D as an index,determines a storage node A memory location based on the key share IDx3, and retrieves the key share x3 from a memory of the storage node atthe storage node memory location.

The method continues at step 194 where the processing module generates asignature contribution based on the key share and a message (e.g., anitem) to sign m. The processing module obtains the message to sign m byat least one of extracting the message to sign m from the signaturecontribution request and generating a hash of a message payloadextracted from the signature contribution request. The generation of thesignature contribution includes generating the signature contribution inaccordance with an expression signature contribution=m^(x) mod n, wherem is the message to sign or the hash of the message payload, x is theretrieved key share, and n is a public modulus (e.g., a public key). Forexample, the processing module generates the signature contribution inaccordance with signature contribution A=m^(x3) mod n, when theprocessing module is associated with storage node A and the key share isx3. The method continues at step 196 where the processing module outputsthe signature contribution to a requesting entity.

FIG. 7 is a flowchart illustrating another example of generating asignature contribution, which includes similar steps to FIG. 6E. Themethod begins at step 200 where a processing module (e.g., of adispersed storage (DS) unit, of a storage node) receives a signaturecontribution request that includes a payload. The method continues atstep 202 where the processing module logs the signature contributionrequest. The logging includes one or more of extracting requestinformation from the signature contribution request, obtaining a useridentifier (ID), obtaining a vault ID, obtaining a timestamp,aggregating the request information, the user ID, the vault ID, and thetimestamp to produce logging information, and facilitating storing ofthe logging information.

The method continues at step 204 where the processing module determineswhether timing of the signature contribution request compares favorablyto a timing template. For example, the processing module determines thatthe comparison is favorable when a difference between the timestampassociated with the signature contribution request and a timestampassociated with a previous signature contribution request is greaterthan a time threshold of the timing template. The method branches tostep 208 when the processing module determines that the timing of therequest compares unfavorably to the timing template. The methodcontinues to step 206 when the processing module determines that thetiming of the request compares favorably to the timing template.

The method continues at step 206 where the processing module determineswhether the signature contribution request compares favorably to afunctionality template. The determination may be based on one or more ofthe payload, a payload analysis, and a comparison of the payloadanalysis to the functionality template. For example, the processingmodule determines that the request compares favorably to thefunctionality template when the processing module determines that aregistry value of the payload does not conflict with a current registryvalue. As another example, the processing module determines that therequest compares favorably to the functionality template when thepayload is not a certificate authority certificate. As yet anotherexample, the processing module determines that the request comparesfavorably to the functionality template when an Internet protocol (IP)address associated with a requester of the request does not compareunfavorably to an unfavorable IP address list.

The method branches to step 192 of FIG. 6E when the processing moduledetermines that the request compares favorably to the functionalitytemplate. The method continues to step 208 when the processing moduledetermines that the request compares unfavorably to the functionalitytemplate. The method continues at step 208 where the processing moduleoutputs a request rejection message. The outputting includes generatingthe request rejection message to include one or more of the signaturecontribution request, the logging information, the timestamp associatedwith the signature contribution request, and an error code, and sendingthe request rejection message to one or more of a requester, a dispersedstorage (DS) imaging unit, a DS processing unit, a DS unit, and a userdevice. When the processing module determines that the request comparesfavorably to the functionality template, the method continues with steps192-196 of FIG. 6E where the processing module retrieves a key sharebased on sharing function parameters, generates a signature result basedon the key share and message to sign, and outputs the signature result.

FIG. 8 is a flowchart illustrating an example of replicating encodeddata slices. The method begins at step 210 where a processing module(e.g., of a dispersed storage (DS) unit) determines whether a frequencyof slice access of an encoded data slice compares favorably to a sliceaccess threshold. The processing module may obtain the frequency ofslice access based on one or more of a frequency of slice access query,a lookup, a list, an error message, a request, and a command. Forexample, the processing module determines that the frequency of sliceaccess compares unfavorably to the slice access threshold when thefrequency of slice access is 500 accesses per minute and the sliceaccess threshold is 100 accesses per minute. The method loops at step210 when the processing module determines that the frequency of accesscompares favorably to the slice access threshold. The method continuesto step 212 when the processing module determines that the frequency ofaccess compares unfavorably to the slice access threshold.

The method continues at step 212 where the processing module determinesat least one secondary DS unit. The determination may be based on one ormore of a current access performance level, a performance requirement,an estimated access performance level, a request pattern, a candidatesecondary DS unit list, a DS unit location, a DS unit performance level,and a DS unit Internet protocol (IP) address. For example, theprocessing module determines the at least one secondary DS unit toinclude a West Coast DS unit went the request pattern includes WestCoast slice access requests and a DS unit performance level associatedwith the West Coast DS unit compares favorably to an access latencyperformance requirement.

The method continues at step 214 where the processing module generates areplicated encoded data slice of the encoded data slice. The generationincludes one or more of immediately retrieving the encoded data slice,retrieving the encoded data slice when a dispersed storage network (DSN)activity level compares favorably to an activity level threshold,rebuilding the encoded data slice, and forming the replicated encodeddata slice from the encoded data slice such that the replicated encodedis substantially the same as the encoded data slice.

The method continues at step 216 where the processing module sends thereplicated encoded data slice to the at least one secondary DS unit forstorage therein. Alternatively, or in addition to, the processing moduledetermines whether the replicated encoded data slice is already storedin the at least one secondary DS unit and sends the replicated encodeddata slice to the at least one secondary DS unit when the replicatedencoded data slice is not already stored in the at least one secondaryDS unit. The method continues at step 218 where the processing moduleupdates a slice storage location table to include an identifierassociated with the at least one secondary DS unit. The updatingincludes updating the slice storage location table to includeaffiliating a slice name associated with the encoded data slice with theidentifier associated with the at least one secondary DS unit.

The method continues at step 220 where the processing module determineswhether the frequency of slice access of the encoded data slice comparesfavorably to a second slice access threshold. For example, theprocessing module determines that the frequency of slice access comparesunfavorably to the second slice access threshold when the frequency ofslice access is greater than the second slice access threshold. Themethod loops at step 220 when the processing module determines that thefrequency of slice access compares unfavorably to the second sliceaccess threshold. The method continues to step 222 when the processingmodule determines that the frequency of slice access compares favorablyto the second slice access threshold.

The method continues at step 222 where the processing module updates theslice storage location table to exclude at least one of the at least onesecondary DS units. The updating includes updating the slice storagetable to exclude affiliating the slice name associated with the encodeddata slice with an identifier associated with at least one of the atleast one secondary DS unit. The at least one secondary DS unit maydelete the replicated encoded data slice when a time period since a lastreplicated encoded data slice access is greater than a deletion timethreshold. The method repeats back to step 210.

FIG. 9A is a schematic block diagram of another embodiment of acomputing system that includes a device 230 (e.g., a user device 12, adispersed storage (DS) processing unit 16) and a dispersed storagenetwork (DSN) memory 36. The DSN memory 36 includes at least one set ofDS units 22. The device 230 includes a module 232 and a pre-fetchsegment buffer 234. The buffer 234 may be implemented utilizing one ormore memory devices, wherein each memory device includes at least one ofa magnetic drive memory, a solid-state memory, and an optical drivememory. The module 232 includes a receive request module 236, a processrequest module 238, a determine buffering module 240, and a pre-fetchmodule 242.

The receive request module 236 is operable to receive a data segmentretrieval request 244 regarding a data segment 246 of a plurality ofdata segments, wherein the data segment 246 is encoded in accordancewith a dispersed storage error coding function (e.g., one or more of thefunctions, processes, etc. discussed with reference to FIG. 4, an IDA,or other type of FEC), to produce a set of encoded data slices, which isstored in the set of DS units 22 of the DSN memory 36. In response tothe data segment retrieval request 244, the process request module 238is operable to process the data segment retrieval request 244. Theprocess request module 238 functions to process the data segmentretrieval request 244 by determining whether the data segment 246 isstored in the pre-fetch segment buffer 234. When the data segment 246 isnot stored in the pre-fetch segment buffer 234, the module 238determines whether the data segment 246 is identified in previouspre-fetch segment buffering information (e.g., a pre-fetch may be inprogress). When the data segment 246 is identified in the previouspre-fetch segment buffering information, the module 238 retrieves thedata segment 246 from the pre-fetch segment buffer 234. When the datasegment 246 is not identified in the previous pre-fetch segmentbuffering information, the module 238 generates a set of at least adecode threshold number of encoded data slice retrieval requests 248regarding at least a decode threshold number of the set of encoded dataslices 250 (e.g., for sending to the set of DS units 22). When encodeddata slices are received, the module 238 decodes at least a decodethreshold number of the encoded data slices 250 (e.g., received from theset of DS units 22) to reproduce the data segment 246.

The pre-fetch segment buffering information 252 includes one or more ofidentity of the one or more other data segments, a number of encodeddata slices to retrieve for each of the one or more data segments,pillar identifiers (IDs) corresponding to encoded data slices of thenumber of encoded data slice to retrieve, a set of DS unit IDsassociated with the pillar IDs, and a data segment retrieval performancegoal. The process request module 238 further functions to process thedata segment retrieval request 244 by determining whether the datasegment 246 is stored in the pre-fetch segment buffer 234 and when thedata segment 246 is stored in the pre-fetch segment buffer 234,retrieving the data segment 246 from the pre-fetch segment buffer 234.

The determine buffering module 240 is operable to determine thepre-fetch segment buffering information 252 based on the data segment246, content of the pre-fetch segment buffer 234, a data consumptionrate, and a DS unit response rate. The DS unit response rate includes aplurality of DS response characteristics for a plurality of DS units,wherein the plurality of DS units includes the set of DS units andwherein a DS response characteristic includes one or more of a pillaridentifier (ID), response latency information, processing consumption,network traffic, response history, and reliability information. Forexample, the determine buffering module 240 determines the pre-fetchsegment buffering information 252 to include an indication to retrieveslices stored in a first and second DS unit for three segments of theone or more other data segments and an indication to retrieve slicesstored in a third DS unit for six segments of the one or more datasegments when response latency information indicates a slower thanaverage response latency for the third DS unit and an average responselatency for the first and second DS units.

The determine buffering module 240 further functions to determine thepre-fetch segment buffering information 252 by determining a datasegment retrieval trend of the plurality of data segments anddetermining the pre-fetch segment buffering information 252 furtherbased on the data segment retrieval trend. For example, the determinebuffering module 240 identifies time intervals between sequential datasegment retrieval requests as the data segment retrieval trend. Asanother example, the determine buffering module 240 identifies intervalsbetween decoding of retrieve data segments as the data segment retrievaltrend.

When the pre-fetch segment buffering information 252 indicatespre-fetching one or more other data segments 258 of the plurality ofdata segments, the pre-fetch module 242 is operable to generate one ormore pre-fetch segment retrieval requests for the one or more other datasegments 258 (e.g., for sending to the set of DS units 22), receive, inresponse to the one or more pre-fetch segment retrieval requests, one ormore sets of at least a decode threshold number of encoded data slices256, decode, in accordance with the dispersed storage error codingfunction, the one or more sets of at least a decode threshold number ofencoded data slices 256 to reproduce the one or more other data segments258, and update the pre-fetch segment buffer 234 with the one or moreother data segments 258. The pre-fetch module 242 functions to generatethe one or more pre-fetch segment retrieval requests by generating, fora pre-fetch segment retrieval request of the one or more pre-fetchsegment retrieval requests, at least a decode threshold number ofencoded data slices retrieval requests 260.

For example, the pre-fetch module 242 generates the decode thresholdnumber of encoded data slice retrieval requests 260. The module 242 thensends, substantially simultaneously, the decode threshold number ofencoded data slice retrieval requests 260 to the set of DS units 22. Inresponse to the requests, the module 242 receives a decode thresholdnumber of encoded data slices 256 and decodes them to reproduce anotherdata segment 258. As another example, the pre-fetch module 242 generatesthe decode threshold number of encoded data slice retrieval requests260, sends a first portion of them to a first portion of DS units (e.g.,DS units associated with an above average retrieval latency) at a firstpoint in time, and sends the remaining portion to remaining DS units(e.g., DS units associated with an average retrieval latency) at asecond point in time. In response to the requests, the module 242receives a decode threshold number of encoded data slices 256 anddecodes them to reproduce another data segment 258 for storage in thepre-fetch segment buffer 234.

FIG. 9B is a timing diagram illustrating an example of pre-fetching adata segment where a request timeline 262 indicates receiving a set ofdata segment requests (e.g., receive data segment requests 1-7) and adelivery timeline 264 indicates delivering a set of corresponding datasegments (e.g., deliver data segments 1-7). The set of data segmentrequests may request sequential data segments of a large data filewithout requesting all data segments of the large data file. Forexample, a request for data segment one is received at time t1, arequest for data segment 2 is received at time t2, a request for datasegment 3 is received at time t4, a request for data segment 4 isreceived at time t6, a request for data segment 5 is received at timet7, a request for data segment 6 is received at time t8, and a requestfor data segment 7 is received at time t9.

A method to pre-fetch data segments for delivery is utilized to providea delivery latency improvement. In an example of operation, a newrequest for data segment 1 is received at time t1 and a data segmentretrieval trend is determined that includes anticipation of receiving atleast another incremental request for data segment 2. Pre-fetch segmentbuffering information is determined to initiate retrieval of at leastdata segments 1-2. Retrieval of data segment 1 from a dispersed storagenetwork memory is initiated at time t1 based on the pre-fetch segmentbuffering information. Pre-fetch of data segment 2 is initiated at timet1 based on the pre-fetch segment buffering information. The datasegment 2 request is received at time t2 and the data segment retrievaltrend is updated noting that a request for data segment two has beenreceived but the pre-fetch of data segment two has not yet beencompleted. The pre-fetch segment buffering information is updated toindicate a more aggressive pre-fetch requirement to speed up datasegment retrievals. As such, pre-fetching of data segments 3 and 4 areinitiated at time t2. Data segments 1 and 2 are delivered at time t3 atthe completion of the retrieval of data segment 1 and the pre-fetch ofdata segment 2.

The example continues where the data segment 3 request is received attime t4 and the data segment retrieval trend is updated noting that thelast three data segment requests are evenly separated by seven timeunits. The pre-fetch segment buffering information is updated toindicate that a delivery rate of data segments is approximately caughtup to a rate of receiving the data segment retrieval requests. As such,pre-fetching of data segment 5 is initiated at time t4. Data segment 3is delivered at time t5. An access latency of nine time units isassociated with delivery of data segment 1, an access latency of twotime units is associated with delivery of data segments 2 and 3. Thedata segment 4 request is received at time t6 and data segment 4 isdelivered at time t6 with a zero time unit access latency due to anadaptation in data segment pre-fetching. The data segment retrievaltrend is updated noting that the data segment 4 retrieval request wasreceived five time units after a previous data segment 3 request. Thepre-fetch segment buffering information is updated to indicate that therate of receiving the data segment retrieval request is increasing. Assuch, pre-fetching of data segments 6 and 7 is initiated at time t6 toattempt to maintain a desired zero latency data segment deliveryperformance level. The data segment 5 retrieval request is received attime t7 and data segment 5 is delivered at time t7 with a zero time unitaccess latency. Data segments 6 and 7 are also delivered with a zerotime unit access latency at times t8 and t9. The method to pre-fetch adata segment is discussed in greater detail with reference to FIG. 9C.

FIG. 9C is a flowchart illustrating example of pre-fetching a datasegment. The method begins with step 270 where a processing module(e.g., of a device) receives a data segment retrieval request regardinga data segment of a plurality of data segments, wherein the data segmentis encoded in accordance with a dispersed storage error coding functionto produce a set of encoded data slices, which is stored in a set ofdispersed storage (DS) units of a dispersed storage network (DSN)memory. In response to the data segment retrieval request, the methodcontinues at step 272 where the processing module processes the datasegment retrieval request by determining whether the data segment isstored in a pre-fetch segment buffer. The method branches to step 276when the processing module determines that the data segment is notstored in the pre-fetch segment buffer. The method continues to step 274when the processing module determines that the data segment is stored inthe pre-fetch segment buffer. The method continues at step 274 where theprocessing module retrieves the data segment from the pre-fetch segmentbuffer. The method branches to step 284.

The method continues at step 276 where the processing module determineswhether the data segment is identified in previous pre-fetch segmentbuffering information when the data segment is not stored in thepre-fetch segment buffer. Pre-fetch segment buffering informationincludes one or more of identity of the one or more other data segments,a number of encoded data slices to retrieve for each of the one or moredata segments, pillar identifiers (IDs) corresponding to encoded dataslices of the number of encoded data slice to retrieve, a set of DS unitIDs associated with the pillar IDs, and a data segment retrievalperformance goal. The method branches to step 280 when the processingmodule determines that the data segment is not identified in previouspre-fetch segment buffering information. The method continues to step278 when the processing module determines that the data segment isidentified in previous pre-fetch segment buffering information. Themethod continues at step 278 where the processing module retrieves thedata segment from the pre-fetch segment buffer when the data segment isstored in the pre-fetch segment buffer when the data segment isidentified in the previous pre-fetch segment buffering information. Themethod branches to step 284.

The method continues at step 280 where the processing module generates aset of at least a decode threshold number of encoded data sliceretrieval requests regarding at least a decode threshold number of theset of encoded data slices when the data segment is not identified inthe previous pre-fetch segment buffering information. The methodcontinues at step 282 where the processing module decodes the at leastthe decode threshold number of the set of encoded data slices toreproduce the data segment.

The method continues at step 284 where the processing module determinespre-fetch segment buffering information based on the data segment,content of a pre-fetch segment buffer, a data consumption rate, and DSunit response rate. The DS unit response rate includes a plurality of DSresponse characteristics for a plurality of DS units, wherein theplurality of DS units includes the set of DS units and wherein a DSresponse characteristic includes one or more of: a pillar identifier(ID), response latency information, processing consumption, networktraffic, response history, and reliability information. The determiningthe pre-fetch segment buffering information includes determining a datasegment retrieval trend of the plurality of data segments anddetermining the pre-fetch segment buffering information further based onthe data segment retrieval trend.

When the pre-fetch segment buffering information indicates pre-fetchingone or more other data segments of the plurality of data segments, themethod continues at step 286 where the processing module generates oneor more pre-fetch segment retrieval requests for the one or more otherdata segments. The generating the one or more pre-fetch segmentretrieval requests includes for a pre-fetch segment retrieval request ofthe one or more pre-fetch segment retrieval requests, generating atleast a decode threshold number of encoded data slices retrievalrequests. The method continues at step 288 where the processing modulereceives, in response to the one or more pre-fetch segment retrievalrequests, one or more sets of at least a decode threshold number ofencoded data slices. The method continues at step 290 where theprocessing module decodes, in accordance with the dispersed storageerror coding function, the one or more sets of at least a decodethreshold number of encoded data slices to reproduce the one or moreother data segments. The method continues at step 292 where theprocessing module updates the pre-fetch segment buffer with the one ormore other data segments.

FIG. 10A is a diagram illustrating an example of encoding data thatincludes data 300, an intermediate matrix 302, a column selector 304, agenerator matrix 306, a data selection 308, and a slice matrix 310 and.The data 300 includes a plurality of data bytes. For example, the dataincludes 100,000 bytes b1-b100 k. The intermediate matrix 302 includesmatrix dimensions (e.g., number of rows, number of columns) based on asize of the data and error coding dispersal storage function parameters(e.g., a decode threshold). For example, the intermediate matrix 302includes five rows and 20,000 columns when the error coding dispersalstorage function parameters includes a decode threshold of five and adata size of the data is 100,000 bytes (e.g., columns=data size/decodethreshold=100k/5=20k). The intermediate matrix 302 includes entries ofsequential data bytes of the data 300 in a row-by-row fashion. Forexample, row 1 includes bytes b1-20 k, row 2 includes bytes b20 k+1-b40k, etc.

The generator matrix 306 includes matrix dimensions based on the errorcoding dispersal storage function parameters (e.g., the decodethreshold, a width). For example, the generator matrix 306 includes fivecolumns and eight rows when the decode threshold is five and the widthis eight. The generator matrix 306 includes entries in accordance withan error coding dispersal storage function to produce encoded dataslices such that at least a decode threshold number of encoded dataslices may be utilized to reproduce the data.

The data selection 308 includes matrix dimensions of one by the decodethreshold (e.g., one by five when the decode threshold is five). Thecolumn selector 304 forms entries of the data selection 308 based onselecting data of each column of the intermediate matrix 302 one by one.For example, the column selector selects a second selection of column 2to include bytes b2, b20 k+2, b40 k+2, b60 k+2, and b80 k+2.

The slice matrix 310 includes matrix dimensions of a width number ofrows and a number of columns substantially the same as the number ofcolumns of the intermediate matrix 302. The slice matrix 310 includesentries that form a width number of encoded data slices. Each encodeddata slice of the width number of encoded data slices includes a numberof bytes substantially the same as the number of columns of theintermediate matrix 302. For example, each encoded data slice of thewidth number of encoded data slices includes 20,000 bytes, when thedecode threshold is five and the size of the data is 100,000 bytes.

In an example of operation, sequential bytes of the data 300 forms thedecode threshold number of rows of the intermediate matrix 302 row byrow. The column selector 304 selects one column of the intermediatematrix printer to at a time to produce a data selection 308 of aplurality of data selections. The generator matrix 306 is multiplied byeach data selection 308 of the plurality of data selections to produce acorresponding column of a plurality of columns of the intermediatematrix 302 of the slice matrix 310. For example, b1_1=ab1+b(b20k+1)+c(b40 k+1)+d(b60 k+1)+e(b80 k+1), b1_2=fb1+g(b20 k+1)+h(b40k+1)+i(b60 k+1)+j(b80 k+1), etc, and b1_8=ajb1+ak(b20 k+1)+a1(b40k+1)+am(b60 k+1)+an(b80 k+1) when the column selector selects a firstcolumn. As another example, b2_1=ab2+b(b20 k+2)+c(b40 k+2)+d(b60k+2)+e(b80 k+2), b2_2=fb2+g(b20 k+2)+h(b40 k+2)+i(b60 k+2)+j(b80 k+2),etc, and b2_8=ajb2+ak(b20 k+2)+a1(b40 k+2)+am(b60 k+2)+an(b80 k+2) whenthe column selector 304 selects a second column.

FIG. 10B is a flowchart illustrating an example of encoding data. Themethod begins at step 312 where a processing module (e.g., of adispersed storage module) receives data for storage. The data mayinclude one or more of a data segment, a data object, a data block, anda data file. The method continues at step 314 where the processingmodule generates an intermediate matrix based on the data and errorcoding dispersal storage function parameters. The method continues atstep 316 where the processing module multiplies a column of theintermediate matrix by a generator matrix to produce a correspondingcolumn of a slice matrix.

The method continues at step 318 where the processing module determineswhether to output one or more columns of the slice matrix. Thedetermination may be based on one or more of a memory utilizationindicator, a memory utilization threshold, a processor loadingindicator, a processor loading threshold, a column count, apredetermination, a message, and a command. For example, the processingmodule determines to output three columns of the slice matrix when thememory utilization indicator compares unfavorably to the memoryutilization threshold. The method repeats back to step 316 to produceanother column of the slice matrix when the processing module determinesnot to output one or more columns of the slice matrix. The methodcontinues to step 320 when the processing module determines to outputone or more columns of the slice matrix.

The method continues at step 320 where the processing module outputs theone or more columns for storage when the processing module determinesoutput one or more columns of the slice matrix. Each column of the oneor more columns includes one or more bytes of a corresponding encodeddata slice. For example, a width number of complete encoded data slicesare output when the one or more columns includes all columns of theintermediate matrix. As another example, a width number of incompleteencoded slices are output when the one or more columns includes lessthan all columns of the intermediate matrix.

The outputting includes sending the one or more columns and sliceinformation to a dispersed storage (DS) unit storage set for storagetherein. The slice information includes one or more of a completeencoded data slice indicator, an append indicator (e.g., to indicatewhether incomplete slices may be appended to form complete slices), andat least some of the error coding dispersal storage function parameters.For example, the slice information includes a complete encoded dataslice indicator such that the indicator indicates incomplete encodeddata slices when the one or more columns includes less than all columnsof the intermediate matrix. As another example, the slice informationincludes an append indicator indicating that incomplete slices may beappended to form complete slices when the intermediate matrix isgenerated based on all bytes of the data. As yet another example, theslice information includes an append indicator indicating thatincomplete or complete slices of a second portion of data may not beappended to incomplete or complete slices of a first portion of the datawhen a first intermediate matrix is generated based on the first portionof the data and a second intermediate matrix is generated based on thesecond portion of the data.

The method continues at step 322 where the processing module determineswhether all columns of intermediate matrix have been processed. Themethod repeats back to step 316 to produce a corresponding column of theslice matrix for a next column. The method ends at step 324 when theprocessing module determines that all columns of the intermediate matrixhave been processed.

FIG. 10C is a flowchart illustrating an example of decoding data. Themethod begins at step 326 where a processing module receives a dataretrieval request to retrieve data. The request may include a dataidentifier (ID) and a user ID. The method continues at step 328 wherethe processing module determines whether the data is stored asincomplete encoded data slices. The determination may be based on one ormore of the data ID, a lookup, a query, an encoded data slice retrievalresponse, a message, and obtaining slice information. For example, theprocessing module determines at least one slice name associated with thedata based on generating the slice name based on the data ID and a vaultID corresponding to the user ID. The processing module sends an encodeddata slice retrieval request, wherein the request includes the at leastone slice name. The processing module receives an encoded data sliceretrieval response corresponding to the encoded data slice retrievalrequest. The receiving may include receiving an encoded data sliceand/or receiving slice information. The processing module determinesthat the data is stored as incomplete encoded data slices when theencoded data slice is incomplete and/or when the slice informationindicates that the data is stored as incomplete encoded slices.

The method branches to step 332 when the processing module determinesthat the data is stored as incomplete encoded data slices. The methodcontinues to step 330 when the processing module determines that thedata is not stored as incomplete encoded data slices. The methodcontinues at step 330 where the processing module retrieves a set ofencoded data slices and branches to step 336.

The method continues at step 332 where the process module retrievesincomplete encoded data slices of each encoded data slice of the set ofencoded data slices when the processing module determines that the datais stored as incomplete encoded data slices. The method continues atstep 334 where the processing module aggregates the incomplete encodeddata slices to reproduce each encoded data slice of the set of encodeddata slices. For example, the processing module reproduces a slicematrix such that sequential columns of corresponding incomplete encodeddata slices form the slice matrix. The method continues at step 336where the processing module dispersed storage error decodes the set ofencoded data slices to reproduce the data. The method may repeat to forma plurality of data segments of the data when the data includes theplurality of data segments.

FIG. 11A is a diagram illustrating another example of encoding data,which includes similar elements as discussed with reference to FIG. 10A.The encoding of data includes data 300, an alternative intermediatematrix 340, a column selector 304, a generator matrix 306, a dataselection 342, and a slice matrix 344. The data 300 includes a pluralityof data bytes. The alternative intermediate matrix 340 includes matrixdimensions (e.g., number of rows, number of columns) based on a size ofthe data 300 and error coding dispersal storage function parameters(e.g., a decode threshold). For example, the alternative intermediatematrix 340 includes five rows and 20,000 columns when the error codingdispersal storage function parameters includes a decode threshold offive and a data size of the data is 100,000 bytes (e.g., columns=datasize/decode threshold=100k/5=20k). The alternative intermediate matrix340 includes entries of sequential data bytes of the data 300 in acolumn-by-column fashion. For example, column 1 includes bytes b1-15,column 2 includes bytes b6-10, etc.

The column selector 304 forms entries of the data selection 342 based onselecting data of each column of the intermediate matrix 340 one by one.For example, the column selector 304 selects a second selection ofcolumn 2 to include bytes b6-b10 of the data 300. The slice matrix 344includes matrix dimensions of the width number of rows and a number ofcolumns substantially the same as a number of columns of the alternativeintermediate matrix 340. The slice matrix 344 includes entries that forma width number of encoded data slices. Each encoded data slice of thewidth number of encoded data slices includes a number of bytessubstantially the same as the number of columns of the alternativeintermediate matrix 340.

In an example of operation, sequential bytes of the data 300 forms thecolumns of the alternative intermediate matrix 340 column by column. Thecolumn selector 304 selects one column of the alternative intermediatematrix 340 at a time to produce a data selection 342 of a plurality ofdata selections. The generator matrix 306 is multiplied by each dataselection 342 of the plurality of data selections to produce acorresponding column of a plurality of columns of the alternativeintermediate matrix 340 of the slice matrix 344. For example,b1_1=ab1+bb2+cb3+db4+eb5, b1_2=fb1+gb2+hb3+ib4+jb5, etc, and b1_8=ajb1+akb2+alb3+amb4+anb5 when the column selector 304 selects a firstcolumn. As another example, b2_1=ab6+bb7+cb8+db9+eb10,b2_2=fb6+gb7+hb8+ib9+jb10, etc, and b2_8=ajb6+akb7+alb8+amb9+anb10 whenthe column selector 304 selects a second column.

Utilizing such an alternative intermediate matrix 340 enables appendingincomplete slices of corresponding encoded slices to form completeencoded data slices when the data is partitioned into two or more datapartitions. For example, an a first incomplete encoded data slice ofencoded data slice 1 may be appended to a second incomplete encoded dataslice of encoded data slice 1, when the data includes byte 1-byte100 k+5and the data is partitioned into two partitions which includes a firstpartition including bytes b1-b100 k and a second partition includingbyte b100 k+1-byte100+5.

An apparatus to execute the appending is described in greater detailwith reference to FIG. 11B and methods of appending is described ingreater detail with reference to FIGS. 11C and 11D.

FIG. 11B is a schematic block diagram of another embodiment of acomputing system that includes a dispersed storage (DS) processing unit16 and dispersed storage network (DSN) memory 36. The DS processing unit16 includes a module 350 (e.g., a DS module 34). Alternatively, the DSprocessing unit 16 may be implemented utilizing a DS module 34 of a userdevice 12. The module 350 includes a receive request module 352, adetermine append module 354, an encode module 356, a generate commandsmodule 358, and an output commands module 360. The DSN memory 36includes at least one set of DS units 22. Each DS unit 22 includes amodule 362 (e.g., a DS module 34) and a memory 370. The module 362includes a store slice module 364, a receive append module 366, and anupdate storage module 368. The memory 370 may be implemented as one ormore memory devices utilizing one or more of a solid-state memory, amagnetic drive memory, and an optical drive memory.

The receive request module 352 is operable to receive a request 372 tostore data 374 in the DSN memory 36. The determine append module 354 isoperable to determine whether the data 374 is to be appended to existingdata 376 stored in the DSN memory 22, wherein the existing data 376 isstored in the DSN memory 36 as a set of encoded data slices and whereinthe existing data 376 is encoded using a dispersed storage error codingfunction to produce the set of encoded data slices. The determine appendmodule 354 functions to determine whether the data 374 is to be appendedby at least one of receiving an append instruction with the data 374,interpreting account information associated with an issuer of therequest to store the data 374, and interpreting directory information ofat least one of the data 374 and the existing data 376.

When the data 374 is to be appended, the encode module 356 is operableto encode, using an append dispersed storage error coding function, thedata 374 to produce a set of encoded append data slices 378. The encodemodule 356 is further operable to encode the existing data using 376 thedispersed storage error coding function by arranging data blocks of theexisting data into a plurality of groups of data blocks and on a groupof data block by group of data block basis, matrix-multiplying theplurality of groups of data blocks with a dispersed storage errorencoding matrix to produce a plurality of sets of encoded data sliceportions and wherein an encoded data slice of the set of encoded dataslices includes an encoded data slice portion from each of the pluralityof sets of encoded data slice portions.

The encode module 356 is further operable to encode the data 374 usingthe append dispersed storage error coding function by partitioning thedata into data blocks based on size of the data blocks of the existingdata, arranging the data blocks of the data into a second plurality ofgroups of data blocks, wherein a group of data blocks of the secondplurality of groups of data blocks includes a same number of data blocksas a group of data blocks of the plurality of groups of data blocks, andon the group of data block by group of data block basis,matrix-multiplying the second plurality of groups of data blocks withthe dispersed storage error encoding matrix to produce a secondplurality of sets of encoded data slice portions and wherein an encodedappend data slice of the set of encoded append data slices includes anencoded data slice portion from each of the second plurality of sets ofencoded data slice portions.

The generate commands module 358 is operable to generate a set of appendcommands 380, wherein an append command of the set of append commands380 includes an encoded append data slice of the set of encoded appenddata slices 378 and identity of one of the set of DS units 22 of the DSNmemory 36. The append command further includes at least one of a slicename of a corresponding one of set of encoded data slices, an offsetindicator, and a column identifier of a slice matrix. The outputcommands module 360 is operable to output at least a write thresholdnumber of the set of append commands 380 to at least a write thresholdnumber of the set of DS units 22. For example, the output commandsmodule 360 outputs an append command 384 of the set of append commands380 a DS unit 22 of the set of DS units 22.

The receive request module 352 is further operable to receive seconddata for storage in the DSN memory 36. The determine append module 354is further operable to determine whether the second data is to beappended to existing appended data 376 stored in the DSN memory 36,wherein the existing appended data includes the set of encoded dataslices and the set of encoded append data slices 378. When the seconddata is to be appended, the encode module 356 is further operable toencode, using the append dispersed storage error coding function, thesecond data to produce a second set of encoded append data slices, thegenerate commands module 358 is further operable to generate a secondset of append commands, and the output commands module 360 is furtheroperable to output at least a write threshold number of the second setof append commands to the at least a write threshold number of the setof DS units 22.

The store slice module 364 is operable to store an encoded data slice382 of the set of encoded data slices, wherein the existing data 376 isencoded using a dispersed storage error coding function to produce theset of encoded data slices. The receive append module 366 is operable toreceive the append command 384 that includes an encoded append dataslice 386 of the set of encoded append data slices 378 and aninstruction to append the encoded append data slice 386 to the encodeddata slice 382, wherein the data 374 is encoded using an appenddispersed storage error coding function to produce the set of encodedappend data slices 378.

The update storage module 368 is operable to update storage of theencoded data slice 382 based on the encoded append data slice 386 toproduce an updated encoded data slice 388. The update storage module 368is further operable to interpret the append command 384 to identify aslice name of the encoded append data slice 386, identify the encodeddata slice 382 based on the slice name, append the encoded append dataslice 386 to the encoded data slice 382 to produce the updated encodeddata slice 388, and update a slice table to associate the slice namewith the updated encoded data slice 388. The update storage module 368functions to update storage of the encoded data slice 382 by identifyingthe encoded data slice 382 (e.g., by data identifier (ID), by a slicename, by a pillar number), retrieving the encoded data slice 382 toproduce a retrieved encoded data slice, appending the encoded appenddata slice 386 to the retrieved encoded data slice to produce theupdated encoded data slice 388, and storing the updated encoded dataslice 388. The receive append module 366 is further operable to receivea second append command that includes a second encoded append data sliceof a second set of encoded append data slices and another instruction toappend the second encoded append data slice to the updated encoded dataslice 388, wherein second data is encoded using the append dispersedstorage error coding function to produce the second set of encodedappend data slices and the update storage module 368 is further operableto update storage of the updated encoded data slice based on the secondkey encoded append data slice to produce a second updated encoded dataslice.

FIG. 11C is a flowchart illustrating an example of appending data. Themethod begins at step 400 where a processing module (e.g., of adispersed storage (DS) processing unit, of a user device) receives arequest to store data in dispersed storage network (DSN) memory. Themethod continues at step 402 where the processing module determineswhether the data is to be appended to existing data stored in the DSNmemory, wherein the existing data is stored in the DSN memory as a setof encoded data slices and wherein the existing data is encoded using adispersed storage error coding function to produce the set of encodeddata slices. The encoding the existing data using the dispersed storageerror coding function includes arranging data blocks of the existingdata into a plurality of groups of data blocks and on a group of datablock by group of data block basis, matrix-multiplying the plurality ofgroups of data blocks with a dispersed storage error encoding matrix toproduce a plurality of sets of encoded data slice portions and whereinan encoded data slice of the set of encoded data slices includes anencoded data slice portion from each of the plurality of sets of encodeddata slice portions.

The determining whether the data is to be appended further includes atleast one of receiving an append instruction with the data, interpretingaccount information associated with an issuer of the request to storethe data, and interpreting directory information of at least one of thedata and the existing data. When the data is to be appended, the methodcontinues at step 404 where the processing module encodes, using anappend dispersed storage error coding function, the data to produce aset of encoded append data slices. The encoding the data using theappend dispersed storage error coding function includes partitioning thedata into data blocks based on size of the data blocks of the existingdata, arranging the data blocks of the data into a second plurality ofgroups of data blocks, wherein a group of data blocks of the secondplurality of groups of data blocks includes a same number of data blocksas a group of data blocks of the plurality of groups of data blocks, andon the group of data block by group of data block basis,matrix-multiplying the second plurality of groups of data blocks withthe dispersed storage error encoding matrix to produce a secondplurality of sets of encoded data slice portions and wherein an encodedappend data slice of the set of encoded append data slices includes anencoded data slice portion from each of the second plurality of sets ofencoded data slice portions.

The method continues at step 406 where the processing module generates aset of append commands, wherein an append command of the set of appendcommands includes one or more of an encoded append data slice of the setof encoded append data slices and identity of one of a set of DS unitsof the DSN memory. The append command further includes at least one of aslice name of a corresponding one of set of encoded data slices, anincremental slice revision level indicator, an offset indicator, and acolumn identifier of a slice matrix. The method continues at step 408where the processing module outputs at least a write threshold number ofthe set of append commands to at least a write threshold number of theset of DS units.

The method continues at step 410 where the processing module receivessecond data (e.g., additional data to append) for storage in the DSNmemory. The method continues at step 412 where the processing moduledetermines whether the second data is to be appended to existingappended data stored in the DSN memory, wherein the existing pendingdata includes the set of encoded data slices and the set of encodedappend data slices. When the second data is to be appended, the methodcontinues at step 414 where the processing module encodes, using theappend dispersed storage error coding function, the second data toproduce a second set of encoded append data slices. The method continuesat step 416 where the processing module generates a second set of appendcommands. The method continues at step 418 where the processing moduleoutputs at least a write threshold number of the second set of appendcommands to the at least a write threshold number of the set of DSunits.

FIG. 11D is a flowchart illustrating an example of appending sliceportions. The method begins at step 430 where a processing module (e.g.,of a dispersed storage (DS) unit) stores an encoded data slice of a setof encoded data slices, wherein existing data is encoded using adispersed storage error coding function to produce the set of encodeddata slices. The method continues at step 432 where the processingmodule receives an append command that includes an encoded append dataslice of a set of encoded append data slices and an instruction toappend the encoded append data slice to the encoded data slice, whereindata is encoded using an append dispersed storage error coding functionto produce the set of encoded append data slices.

The method continues at step 434 where the processing module interpretsthe append command to identify one or more of a slice name of theencoded append data slice and an incremental revision level indicator.The method continues at step 436 where the processing module identifiesthe encoded data slice based on one or more of the slice name and theincremental revision level indicator. The method continues at step 438where the processing module appends the encoded append data slice to theencoded data slice to produce the updated encoded data slice. The methodcontinues at step 440 where the processing module updates storage of theencoded data slice based on the encoded append data slice to produce anupdated encoded data slice. The updating storage of the encoded dataslice includes identifying the encoded data slice (e.g., by dataidentifier (ID), by a slice name, by a pillar number, by the incrementalrevision level indicator), retrieving the encoded data slice to producea retrieved encoded data slice, appending the encoded append data sliceto the retrieved encoded data slice to produce the updated encoded dataslice, and storing the updated encoded data slice.

The method continues at step 442 where the processing module updates aslice table to associate the slice name with the updated encoded dataslice. The method continues at step 444 where the processing modulereceives a second append command that includes a second encoded appenddata slice of a second set of encoded append data slices and anotherinstruction to append the second encoded append data slice to theupdated encoded data slice, wherein second data is encoded using theappend dispersed storage error coding function to produce the second setof encoded append data slices. The method continues at step 446 wherethe processing module updates storage of the updated encoded data slicebased on the second encoded append data slice to produce a secondupdated encoded data slice.

FIGS. 12A-E are schematic block diagrams of another embodiment of thecomputing system that each include a dispersed storage (DS) processingunit 16 and DS units 1-5, wherein each Figure of FIGS. 12A-E corresponda step of a Shamir secret share rebuilding scenario. In such a Shamirsecret share rebuilding scenario, a first step includes generating andstoring a set of Shamir secret shares in DS units 1-5, a second stepincludes determining a Shamir secret share of the set of Shamir secretshares to be rebuilt, a third step includes requesting a decodethreshold number of rebuilt Shamir secret share partials, a fourth stepincludes generating the decode threshold number of rebuilt Shamir secretshare partials, and a fifth step includes decoding the decode thresholdnumber of rebuilt Shamir secret share partials to produce a rebuiltShamir secret share and storing the rebuilt Shamir secret share. Themethod of operation is discussed in greater detail with reference toFIGS. 12A-13B.

FIG. 12A is a schematic block diagram of another embodiment of acomputing system of a first step of a Shamir secret share rebuildingscenario, where a dispersed storage (DS) processing unit 16 utilizes aShamir shared secret function on a secret to produce a width number ofShamir secret shares 1-5. The DS processing unit 16 sends the widthnumber of Shamir secret shares 1-5 to corresponding DS units 1-5 forstorage therein.

FIG. 12B is a schematic block diagram of another embodiment of acomputing system of a second step of a Shamir secret sharing rebuildingscenario, where a dispersed storage (DS) processing unit 16 determines aShamir secret share to be rebuilt. For example, the processing modulesends a width number of Shamir secret share retrieval requests to awidth number of DS units 1-5 and determines that a Shamir secret sharecorresponding to DS unit 2 is missing based on receiving Shamir secretshares corresponding to DS units 1, and DS units 3-5 (e.g., and not fromDS unit 2).

FIG. 12C is a schematic block diagram of another embodiment of acomputing system of a third step of a Shamir secret sharing rebuildingscenario, where a dispersed storage (DS) processing unit 16 generates adecode threshold number of partial request messages and sends the decodethreshold number of partial request messages to a decode thresholdnumber of DS units associated with other Shamir secret shares. Thepartial request message of the decode threshold number of partialrequest messages includes an identifier of a Shamir secret share to berebuilt. For example, the processing module generates the decodethreshold number of partial request messages to include an identifier of2 for the Shamir secret share to be rebuilt and sends the decodethreshold number of partial request messages to the decode thresholdnumber of DS units including DS units 1, 3, and 4.

FIG. 12D is a schematic block diagram of another embodiment of acomputing system of a fourth step of a Shamir secret sharing rebuildingscenario, where each dispersed storage (DS) unit of a decode thresholdnumber of DS units generates a rebuilt share partial based on anidentifier of a Shamir secret share to be rebuilt and a correspondinglocal Shamir secret share and sends the rebuilt share partial to a DSprocessing unit 16. For example, DS unit 1 generates the rebuilt sharepartial 2-1 based on an identifier of 2 for the Shamir secret share tobe rebuilt and a corresponding local Shamir secret share 1 and sends therebuilt share partial 2-1 to the DS processing unit 16, DS unit 3generates the rebuilt share partial 2-3 based on the identifier of 2 forthe Shamir secret share to be rebuilt and a corresponding local Shamirsecret share 3 and sends the rebuilt share partial 2-3 to the DSprocessing unit 16, and DS unit 4 generates the rebuilt share partial2-4 based on the identifier of 2 for the Shamir secret share to berebuilt and a corresponding local Shamir secret share 4 and sends therebuilt share partial 2-4 to the DS processing unit 16.

FIG. 12E is a schematic block diagram of another embodiment of acomputing system of a fifth step of a Shamir secret sharing rebuildingscenario, where a dispersed storage (DS) processing unit 16 receives adecode threshold number of rebuilt share partials, decodes the decodethreshold number of rebuilt share partials to produce a rebuilt Shamirsecret share, and stores the rebuilt Shamir secret share in a DS unitcorresponding to a Shamir secret share to be rebuilt. For example, theprocessing unit 16 receives rebuilt share partials 2-1, 2-3, and 2-4,decodes the rebuilt share partials 2-1, 2-3, and 2-4 to produce rebuiltShamir secret share 2, and sends the rebuilt Shamir secret share 2 to DSunit 2 for storage therein.

FIG. 13A is a flowchart illustrating an example of rebuilding a Shamirsecret share. The method begins with step 450 where a processing module(e.g., of a dispersed storage (DS) processing unit) determines a Shamirsecret share to be rebuilt. The determination may be based on one ormore of a query, a Shamir secret share retrieval sequence, an errormessage, and a decode result. The method continues at step 452 where theprocessing module determines which other Shamir secret shares to utilizein a rebuilding function. The determination may be based on one or moreof a DS unit status indicator, a decode threshold, a decode result, aretrieval result, a query, and a message. For example, the processingmodule determines other Shamir secret shares to utilize in therebuilding function to include DS units 1, 3, and 4 when the decodethreshold is three and a corresponding Shamir secret share has beensuccessfully retrieved from each DS unit of DS units 1, 3, and 4.

The method continues at step 454 where the processing module generates adecode threshold number of partial request messages. The messagesinclude one or more of an identifier (ID) associated with the Shamirsecret share to be rebuilt, which other Shamir secret shares are beingutilized in the rebuilding function, and encoding matrix, a squarematrix, an inverted square matrix, and a row of the matrix correspondingto the Shamir secret share to be rebuilt (e.g., wrote 2 for Shamirsecret share 2 to be rebuilt). The method continues at step 456 wherethe processing module sends the decode threshold number of partialrequest messages to a decode threshold number of DS units associatedwith the other Shamir secret shares. For example, the processing modulesends a partial request message that includes a request for rebuiltshare partial 2-1 to DS unit 1, the processing module sends a partialrequest message that includes a request for rebuilt share partial 2-3 toDS unit 3, and the processing module sends a partial request messagethat includes a request for rebuilt share partial 2-4 to DS unit 4.

The method continues at step 458 where the processing module receives adecode threshold number of rebuilt share partials. For example, theprocessing module receives rebuilt share partial 2-1, 2-3, and 2-4. Themethod continues at step 460 where the processing module decodes thedecode threshold number of rebuilt share partials to produce a rebuiltShamir secret share. For example, the processing module decodes thedecode threshold number of rebuilt share partials 2-1, 2-3, and 2-4 inaccordance with the formula rebuilt Shamir secret share 2=(rebuilt sharepartial 2-1) exclusive OR (XOR) (rebuilt share partial 2-3) XOR (rebuiltshare partial 2-4). The method continues at step 462 where theprocessing module outputs the rebuilt Shamir secret share to a DS unitfor storage therein, wherein the DS unit is associated with the Shamirsecret share to be rebuilt. For example, the processing module sendsrebuilt Shamir secret share 2 to DS unit 2 for storage therein.

FIG. 13B is a flowchart illustrating an example of generating a rebuiltshare partial. The method begins at step 464 where processing module(e.g., of a dispersed (DS) storage unit) receives a partial requestmessage to provide a rebuilt share partial. The method continues at step466 where the processing module generates an inverted square matrix. Thegeneration includes one or more of receiving the inverted square matrix,obtaining an encoding matrix (e.g., extracted from the request message,retrieved), selecting a decode threshold number of rows of the encodingmatrix to produce a square matrix (e.g., corresponding to other Shamirsecret shares to be utilized in a rebuilding function), and invertingthe square matrix to produce the inverted square matrix.

The method continues at step 468 where the processing module retrieves alocal Shamir secret share. The retrieving includes identifying the localShamir secret share to produce a local Shamir secret share identifier(ID) based on the partial request message and retrieving the localShamir secret share based on the local Shamir secret share ID. Forexample, the processing module identifies the local Shamir secret shareas local Shamir secret share 1 based on receiving a request message forrebuilt share partial 2-1 and retrieves Shamir secret share 1 from alocal memory of a dispersed storage (DS) unit.

The method continues at step 470 where the processing module obtains arow of a full encoding matrix associated with the Shamir secret share tobe rebuilt as a target row. The obtaining may include one or more ofreceiving the target row in the partial request message and extractingthe target row from the full encoding matrix based on an ID of theShamir secret share to be rebuilt. The processing module may obtain thefull encoding matrix based on one or more of extracting the fullencoding matrix from the partial request message and retrieving the fullencoding matrix. For example, the processing module extracts row 2 ofthe full encoding matrix when the request message includes a request forrebuilt share partial 2-1.

The method continues at step 472 where the processing module generatesthe rebuilt share partial based on the inverted square matrix, the localShamir secret share, and the target row. For example, the processingmodule multiplies the inverted square matrix by the local Shamir secret(e.g., in matrix form wherein the local Shamir secret share is placed ina row corresponding to an ID of the local Shamir secret share) by thetarget row to produce the rebuilt share partial. The method continues atstep 474 where the processing module outputs the rebuilt share partial.For example, the processing module sends the rebuilt share partial to arequesting entity (e.g. a DS processing unit), associated with thepartial request message.

FIGS. 14A-D are schematic block diagrams of another embodiment of acomputing system that each include a dispersed storage (DS) processingunit 16 and DS units 1-6, wherein each figure of FIGS. 14A-D corresponda step of a data update scenario. In such a data update scenario, afirst step includes the DS processing unit 16 generating and storingdata as data 1-4 and parity of the data as parity 5-6 in DS units 1-6when an associated encoding function includes a decode threshold of 4and a width of 6, a second step includes the DS processing unit 16storing updated data as updated data 2 by sending updated data 2 to DSunit 2 for storage therein, a third step includes DS unit 2 generatingdelta parity 5 and delta parity 6 based on data 2 and updated data 2 andsending delta parity 5 and delta parity 6 to DS processing unit 16, anda fourth step includes the DS processing unit 16 forwarding delta parity5 to DS unit 5 and delta parity 6 to DS unit 6, DS unit 5 generates andstores updated parity 5 based on parity 5 and delta parity 5, and DSunit 6 generates and stores updated parity 6 based on parity 6 and deltaparity 6.

Alternatively, the third step includes the DS processing unit 16retrieving data 2 from DS unit 2 and generating delta parity 5 and deltaparity 6 based on data 2 and updated data 2. Alternatively, the fourthstep includes the DS processing unit 16 retrieving parity 5 from DS unit5 and retrieving parity 6 from DS unit 6, generating updated parity 5and updated parity 6, and sending updated parity 5 to DS unit 5 forstorage therein and sending updated parity 6 to DS unit 6 for storagetherein. The method of operation is discussed in greater detail withreference to FIGS. 14A-15C.

FIG. 14A is a schematic block diagram of another embodiment of acomputing system of a first step of a data update scenario, where adispersed storage (DS) processing unit 16 generates and stores data asdata 1-4 and parity of the data as parity 5-6 in DS units 1-6. Thegeneration of data 1-4 and parity 5-6 is in accordance with a dataencoding function, wherein the data encoding function includes utilizingan encoding matrix. In a first embodiment, the encoding matrix includesa unity square matrix (e.g., a first decode threshold number of rowseach includes a one in a single column of a decode threshold number ofcolumns producing a diagonal string of one's) and includes a widthnumber minus the decode threshold number of parity rows. The parity rowsinclude encoding matrix entries in accordance with the data encodingfunction. In a second embodiment, encoding matrix does not include theunity square matrix.

The generation of data 1-4 and parity 5-6 includes matrix multiplyingthe encoding matrix by the data to produce a width number of encodeddata slices, wherein encoded data slices 1-4 produce data 1-4 andencoded data slices 5-6 produce parity 5-6. The storing of the dataincludes sending data 1 to DS unit 1 for storage therein, sending data 2to DS unit 2 for storage therein, sending data 3 to DS unit 3 forstorage therein, sending data 4 to DS unit 4 for storage therein,sending parity 5 to DS unit 5 for storage therein, and sending parity 6to DS unit 6 for storage therein.

FIG. 14B is a schematic block diagram of another embodiment of acomputing system of a second step of a data update scenario, where adispersed storage (DS) processing unit 16 obtains an updated portion ofdata to produce updated data and sends an updated data storage requestmessage to a corresponding DS unit to replace a corresponding portion ofthe data. The obtaining may include receiving the updated portion ofdata, receiving data that includes the updated portion of data, andanalyzing data to identify the updated portion of data. The storagerequest message may include one or more of the updated data, encodinginformation, and a request for delta parity information. The encodinginformation may include one or more of an encoding matrix, a width, anda decode threshold. For example, the DS processing unit 16 receives anupdated portion of data corresponding to a second portion of data toproduce updated data 2 and sends a storage request message to DS unit 2that includes the updated data 2.

FIG. 14C is a schematic block diagram of another embodiment of acomputing system of a third step of a data update scenario, where adispersed storage (DS) unit generates delta parity information based onupdated data, data (e.g., previous data which is being replaced by theupdated data), and an encoding function and sends the delta parityinformation to a corresponding DS processing unit 16. Alternatively, theDS unit directly sends the delta parity information to one or more otherDS units associated with storage of parity information. For example, DSunit 2 generates the delta parity information to include delta parity 5and delta parity 6 based on updated data 2, data 2, and the encodingfunction and sends the delta parity information to the DS processingunit 16. For instance, DS unit 2 generates the delta parity informationin accordance with formulas delta parity 5=rebuilt partial (for parity 5based on updated data 2)−rebuilt partial (for parity 5 based on data 2)and delta parity 6=rebuilt partial (for parity 6 based on updated data2)−rebuilt partial (for parity 6 based on data 2) and sends the deltaparity information to a corresponding DS processing unit 16. The DS unit2 generates such a rebuilt partial (for parity 5 based on updated data2) by multiplying an inverted square matrix of a unity matrix of anencoding matrix of the encoding function by a data matrix including theupdated data 2 by a row of the encoding matrix corresponding to parity5.

FIG. 14D is a schematic block diagram of another embodiment of acomputing system of a fourth step of a data update scenario, where adispersed storage (DS) processing unit 16 outputs delta parityinformation to one or more DS units associated with storing parityinformation and each of the one or more DS units generates and storesupdated parity information based on the parity information, storedparity information, and encoding information. For example, the DSprocessing unit 16 outputs delta parity 5 to DS unit 5 and delta parity6 to DS unit 6. DS unit 5 retrieves parity 5 (e.g., from a local DS unit5 memory), wherein parity 5 is associated with updated parity 5. DS unit5 generates updated parity 5 in accordance with a formula updated parity5=parity 5+delta parity 5. DS unit 5 stores updated parity 5 (e.g., inthe local memory), replacing parity 5. DS unit 6 retrieves parity 6(e.g., from a local DS unit 6 memory), wherein parity 6 is associatedwith updated parity 5. DS unit 6 generates updated parity 6 inaccordance with a formula updated parity 6=parity 6+delta parity 6. DSunit 6 stores updated parity 6 (e.g., in the local DS unit 6 memory),replacing parity 6.

FIG. 15A is a schematic block diagram of an embodiment of a dispersedstorage system that includes an update slice module 480, a generateerror recovery information module 482, and an update error recoveryinformation module 484. The update slice module 480, when operablewithin a computing device, causes the computing device to update anencoded data slice 490 of a set of encoded data slices to produce anupdated encoded data slice 486, wherein data is encoded in accordancewith a dispersed storage error coding function to produce the set ofencoded data slices and wherein at least some of the encoded data slicesof the set of encoded data slices include error recovery information.For example, the update slice module 480 receives updated data,determines an identity of the encoded data slice 490 based on at leastone difference between the data and the updated data, and generates theupdated encoded data slice 486 based on the difference and in accordancewith the dispersed storage error coding function.

The generate error recovery information module 482, when operable withina second computing device, causes the second computing device to, foreach one of the at least some of the encoded data slices that includeerror recovery information, generate partial error recovery informationbased on one or more of: the encoded data slice 490, the updated encodeddata slice 486, update information, and the dispersed storage errorcoding function to produce a collection of partial error recoveryinformation 488. The update information includes one or more of a slicename, a revision number of the updated encoded data slice, a generatormatrix, an encoding matrix, a pillar number of the updated encoded dataslice.

The generate error recovery information module 482 functions to generateone of the collection of partial error recovery information bygenerating a first term (e.g., a partially encoded data slice utilizingthe updated encoded data source 486) of a corresponding one of the atleast some of the encoded data slices that includes the error recoveryinformation based on the updated encoded data slice 486 in accordancewith the dispersed storage error coding function, generating a secondterm (e.g., another partially encoded data slice utilizing the encodeddata slice 490) of the corresponding one of the at least some of theencoded data slices that includes the error recovery information basedon the encoded data slice 490 in accordance with the dispersed storageerror coding function, and performing a mathematical function on thefirst and second terms to produce the one of the collection of partialerror recovery information. For example, the generate error recoveryinformation module 482 adds the first and second terms to produce theone of the collection of partial error recovery information. As anotherexample, the generate error recovery information module 482 performs anexclusive OR logical function on the first and second terms to producethe one of the collection of partial error recovery information.

The generate error recovery information module 482 functions to generatethe first term by obtaining an encoding matrix of the dispersed storageerror coding function (e.g., retrieve from local memory, receive),reducing the encoding matrix to produce a square matrix that excludesone or more rows including a row associated with the corresponding oneof the at least some of the encoded data slices that includes the errorrecovery information (e.g., alternatively may receive the squarematrix), inverting the square matrix to produce an inverted matrix(e.g., alternately may receive the inverted matrix), matrix multiplyingthe inverted matrix by the updated encoded data slice 486 to produce avector, and matrix multiplying the vector by a row of the encodingmatrix corresponding to the corresponding one of the at least some ofthe encoded data slices that includes the error recovery information toproduce the first term. The generate error recovery information module482 functions to generate the second term by matrix multiplying theinverted matrix by the encoded data slice 490 to produce a second vectorand matrix multiplying the second vector by the row of the encodingmatrix corresponding to the corresponding one of the at least some ofthe encoded data slices that includes the error recovery information toproduce the second term.

The update error recovery information module 484, when operable withinthe first or the second computing device, causes the first or the secondcomputing device to update the error recovery information of an encodeddata slice 492 of the at least some of the encoded data slices thatinclude error recovery information based on a corresponding one of thecollection of partial error recovery information 488 to produce anencoded data slice that includes updated error recovery information 494.The update error recovery information module 484 functions to update theerror recovery information by executing a mathematical function on theencoded data slice that includes the error recovery information 492 andthe corresponding one of the collection of partial error recoveryinformation 488 to produce the encoded data slice that includes updatederror recovery information 494. For example, update error recoveryinformation module 484 adds the encoded data slice that includes theerror recovery information 492 and the corresponding one of thecollection of partial error recovery information 488 to produce theencoded data slice that includes updated error recovery information 494.As another example, update error recovery information module 484performs an exclusive OR function on the encoded data slice thatincludes the error recovery information 492 and the corresponding one ofthe collection of partial error recovery information 488 to produce theencoded data slice that includes updated error recovery information 494.

FIG. 15B is a schematic block diagram of another embodiment of acomputing system that includes a dispersed storage (DS) processing unit16, a computing device 501 (e.g., a DS unit 22, a user device, anotherDS processing unit), and a set of DS units 22. The computing device 501includes a DS module 500 and a memory 502. The memory 502 may beimplemented utilizing one or more memory devices, wherein each memorydevice includes at least one of a magnetic drive memory, a solid-statememory, and an optical drive memory. The DS module 500 includes areceive slice module 504, a generate error recovery information module482, and an output error recovery information module 506.

The receive slice module 504, when operable within the computing device501, causes the computing device to facilitate receiving an updatedencoded data slice 486, wherein the updated encoded data slice 486 is anupdated version of an encoded data slice 490 of a set of encoded dataslices, wherein data is encoded in accordance with a dispersed storageerror coding function to produce the set of encoded data slices andwherein at least some of the encoded data slices of the set of encodeddata slices include error recovery information. The receive slice module504 is further operable to facilitate storing the updated encoded dataslice 486 (e.g., in memory 502).

The generate error recovery information module 482, when operable withinthe computing device 501, causes the computing device to, for each oneof the at least some of the encoded data slices that include errorrecovery information, generate partial error recovery information 508based on one or more of: the encoded data slice 490, the updated encodeddata slice 486, update information, and the dispersed storage errorcoding function to produce a collection of partial error recoveryinformation 488. The update information includes one or more of a slicename, a revision number of the updated encoded data slice, a generatormatrix, an encoding matrix, a pillar number of the updated encoded dataslice. The output error recovery information module 506, when operablewithin the computing device 501, causes the computing device tofacilitate outputting the collection of partial error recoveryinformation 488 for storage in at least some of the set of DS units 22,which store the at least some of the encoded data slices that includeerror recovery information. For example, the output error recoveryinformation module 506 outputs partial error recovery information 508 toa fifth DS unit, wherein the fifth DS unit corresponds to one of the atleast some of the encoded data slices of the set of encoded data slicesthat includes error recovery information and the partial error recoveryinformation 508 corresponds to the fifth DS unit.

FIG. 15C is a flowchart illustrating an example of updating errorrecovery information. The method begins at step 520 where a dispersedstorage (DS) processing module (e.g., of a DS processing unit) updatesan encoded data slice of a set of encoded data slices to produce anupdated encoded data slice, wherein data is encoded in accordance with adispersed storage error coding function to produce the set of encodeddata slices and wherein at least some of the encoded data slices of theset of encoded data slices include error recovery information. Themethod continues at step 522 where the DS processing module sends theupdated encoded data slice to a first DS unit of a set of DS units. Themethod continues at step 524 where the DS processing module generatesupdate information to include one or more of: a slice name, a revisionnumber of the updated encoded data slice, a generator matrix, anencoding matrix, a pillar number of the updated encoded data slice. Themethod continues at step 526 where the DS processing module sends theupdate information to the first DS unit.

The method continues at step 528 where the first DS unit stores theupdated encoded data slice. In addition, the first DS unit stores theupdate information when the first DS unit receives the updateinformation. For each one of the at least some of the encoded dataslices that include error recovery information, the method continues atstep 530 where the first DS unit generates partial error recoveryinformation based on one or more of: the encoded data slice, the updatedencoded data slice, update information, and the dispersed storage errorcoding function to produce a collection of partial error recoveryinformation. The generating the one of the collection of partial errorrecovery information includes generating a first term of a correspondingone of the at least some of the encoded data slices that includes theerror recovery information based on the updated encoded data slice inaccordance with the dispersed storage error coding function, generatinga second term of the corresponding one of the at least some of theencoded data slices that includes the error recovery information basedon the encoded data slice in accordance with the dispersed storage errorcoding function, and performing a mathematical function (e.g.,subtraction, exclusive OR) on the first and second terms to produce theone of the collection of partial error recovery information.

The generating the first term includes obtaining an encoding matrix ofthe dispersed storage error coding function, reducing the encodingmatrix to produce a square matrix that excludes one or more rowsincluding a row associated with the corresponding one of the at leastsome of the encoded data slices that includes the error recoveryinformation, inverting the square matrix to produce an inverted matrix,matrix multiplying the inverted matrix by the updated encoded data sliceto produce a vector, and matrix multiplying the vector by a row of theencoding matrix corresponding to the corresponding one of the at leastsome of the encoded data slices that includes the error recoveryinformation to produce the first term. The generating the second termincludes matrix multiplying the inverted matrix by the encoded dataslice to produce a second vector and matrix multiplying the secondvector by the row of the encoding matrix corresponding to thecorresponding one of the at least some of the encoded data slices thatincludes the error recovery information to produce the second term.

The method continues at step 532 where the first DS unit outputs thecollection of partial error recovery information for storage (e.g., as aportion of updated encoded data slices) in at least some of the set ofDS units, which store the at least some of the encoded data slices thatinclude error recovery information. The method continues at step 534where one of the at least some of the set of DS units updates the errorrecovery information of an encoded data slice of the at least some ofthe encoded data slices that includes the error recovery informationbased on a corresponding one of the collection of partial error recoveryinformation to produce an encoded data slice that includes updated errorrecovery information. The updating the error recovery informationincludes executing a mathematical function (e.g., summing, exclusive OR)on the encoded data slice that includes the error recovery informationand the corresponding one of the collection of partial error recoveryinformation to produce the encoded data slice that includes updatederror recovery information. Next, the one of the at least some of theset of DS units stores the encoded data slice that includes updatederror recovery information.

FIG. 16 is a diagram illustrating an example of a directory filestructure that includes directory files 1-3. Alternatively, any numberof directory files may be included. The directory files 1-3 may beutilized to affiliate file system filenames to storage locations withina dispersed storage network (DSN) memory. The storage location may bespecified by a source name within the DSN memory. The source name mayinclude one or more of a vault identifier (ID), a generation ID, and anobject number. The object number may include a random number that ispermanently assigned to data to be stored in the DSN memory upon a firststorage sequence of the data. A vault source name includes a source nameand a data segment ID.

Each directory file of the directory files 1-3 may be stored as encodeddirectory slices in the DSN memory at a location affiliated with thedirectory file. For example, directory file 1 is dispersed storage errorencoded to produce one or more sets of encoded directory 1 slices thatare stored in the DSN memory at location source name 1 (e.g., B530). Asanother example, directory file 2 is dispersed storage error encoded toproduce one or more sets of encoded directory 2 slices that are storedin the DSN memory at location source name 2 (e.g., 42DA). As yet anotherexample, directory file 3 is dispersed storage error encoded to produceone or more sets of encoded directory 3 slices that are stored in theDSN memory at location source name 3 (e.g., E9C2).

Each directory file of the directory files 1-3 includes a file namefield 540, a file source name field 542, a snapshot field 544, anextended data field 546, and a linked directory source names field 548.Each field of the directory file includes one or more entries, whereineach entry of the one or more entries per field is associated with anentry within each other field of a common row of the directory file. Thefile name field 540 includes one or more entries, wherein each entry ofthe one or more entries includes a file system file name including atleast one of a root directory name, a directory name, and a file name.For example, a directory name entry of the file name field includes/lists and a file name entry of the file name field includes /file.docand /pic2.jpg.

The file source name field 542 includes one or more entries, whereineach entry of the one or more entries includes a source name of acorresponding entry (e.g., same row) in the file name field. Forexample, a file source name field entry of B673 associated with a filename field entry of /file.doc indicates that the file with file name/file.doc is stored in the DSN memory (e.g., as a plurality of sets ofencoded data slices) at a location with a source name of B673. Asanother example, a file with file name /pic2.jpg is stored in the DSNmemory at a location with a source name of 7AA7. As yet another example,a directory file with directory name /lists is stored in the DSN memoryat a location with a source name of 90DE. Accessing such a directoryfile associated with /lists may be utilized to access one or more filesunder the directory /lists. For example, accessing the directory filestored in the DSN memory at the location with the source name of 90DEmay be utilized to access a file associated with a file name of/lists/summary.doc. As another example, accessing the directory filestored in the DSN memory at the location with the source name of 90DEmay be utilized to access a sub-directory of /lists/documents andaccessing the sub-directory of /lists/documents may be utilized access afile associated with a file name of /lists/documents/reportA.doc. Assuch, the directory file structure may be associated with any number oflevels (e.g., sub-directories).

The snapshot field 544 includes one or more entries, wherein each entrythe one or more entries includes a snapshot ID of a corresponding entry(e.g., same row) in the file name field. For example, a snapshot fieldentry of 1 associated with the file name field entry of /file.docindicates that the file with file name /file.doc is associated with asnapshot ID of 1. As another example, the file with file name /pic2.jpgis associated with a snapshot ID of 2. As yet another example, thedirectory file with directory name /lists is associated with a snapshotID of 5.

The extended data field 546 includes one or more entries, wherein eachentry of the one or more entries includes at least one of a timestamp, asize indicator, a segmentation allocation table (SAT) vault source name,metadata, and a content portion associated with a corresponding entry(e.g., same row) in the file name field. For example, an extended datafield entry of 329 d associated with the file name field entry of/file.doc indicates that the file with file name /file.doc is associatedwith an extended data value of 329 d. As another example, the file withfile name /pic2.jpg is associated with an extended data value of a401.As yet another example, the directory file with directory name /lists isassociated with an extended data value of fb79.

The linked directory source names field 548 includes one or moreentries, wherein each entry the one or more entries includes zero ormore source names of linked directory files associated with acorresponding entry (e.g., same row) in the file name field and/or acorresponding entry in the snapshot field. For example, a linkeddirectory source names field entry of 42DA associated with the file namefield entry of /file.doc indicates that the file with file name/file.doc and snapshot ID 1 is associated with a linked directory filewith a DSN address of 42DA. As another example, the file with file name/pic2.jpg and snapshot ID 2 is associated with the linked directory filewith the DSN address of 42DA and is associated with a linked directoryfile with a DSN address of E9C2. As yet another example, the directoryfile with directory name /lists is not associated with a linkeddirectory file.

The linked directory source name field 548 provides linkage between twoor more portions of the directory file structure. The linkage may beutilized when directory files include affiliated entries. Theaffiliation includes entries that share common filenames with differentsnapshot IDs, entries that share common filenames with differentrevisions, entries of file names that are moved from a first directoryto a second directory, and entries of filenames that are cloned from afirst directory to a second directory. For example, a second revision offile name /pic2.jpg of a second snapshot included in directory file 1 islinked to a first revision of file name /pic.jpg of a first snapshotincluded in directory file 2 and is linked to a third revision of filename /pic3/jpg of a third snapshot included in directory file 3. Asanother example, a first revision of file name /file.doc of a firstsnapshot included in directory file 1 is linked to a second revision offile name /file2.doc of a second snapshot included in directory file 2.

A request to delete a file may result in deletion of an associateddirectory file entry and in deletion of encoded data slices associatedwith the file in accordance with a deletion method. The deletion methodmay be based on one or more of a snapshot ID associated with a file nameof the file from a primary directory file and one or more associatedsnapshot IDs and corresponding filenames from one or more linkeddirectory files (e.g., utilizing one or more linked directory sourcenames from the primary directory file).

For example, a plurality of encoded data slices associated with filename /file2.doc at source name B775 are deleted, a plurality of encodeddata slices associated with file name /file.doc at source name B673 aredeleted, a directory file 2 entry associated with file name /file2.docis deleted, and a directory file 1 entry associated with file name/file.doc is deleted when a request is received to delete the fileassociated with the file name /file2.doc since file name/file2.doc isassociated with a snapshot ID of 2, only one linked directory exists(e.g., directory file 1), an associated entry of linked directory file 1for file name /file.doc is associated with a snapshot ID of 1 (e.g.,older), and the deletion method specifies to delete older snapshots whena newer snapshot is deleted.

As another example, the directory file 1 entry associated with file name/file.doc is deleted when a request is received to delete the fileassociated with the file name /file.doc since file name/file.doc isassociated with a snapshot ID of 1, only one linked directory exists(e.g., directory file 2), an associated entry of linked directory file 2for file name /file2.doc is associated with a snapshot ID of 2 (e.g.,newer), and the deletion method specifies to not delete newer snapshotsand associated older snapshots one and older snapshot is deleted. Themethod to process a request to delete a file is discussed in greaterdetail with reference to FIG. 17.

FIG. 17 is a flowchart illustrating an example of deleting a snapshot.The method begins at step 550 where a processing module receives adelete snapshot request. The delete snapshot request includes one ormore of a snapshot identifier (ID), a file name, a primary directorysource name, and a vault ID. The method continues at step 552 where theprocessing module accesses an entry of a primary directory correspondingto the snapshot ID. The accessing includes one or more of obtaining(e.g., receiving, traversing a directory structure, a query) a sourcename of the primary directory, retrieving at least one set of encodedprimary directory slices from a dispersed storage network (DSN) memory,dispersed storage error decoding the at least one set of encoded primarydirectory slices to produce a primary directory file, identifying anentry of the primary directory file corresponding to the snapshot IDand/or the file name, and extracting the entry of the primary directoryfile.

The method continues at step 554 where the processing module determineswhether there are one or more linked secondary directories. Thedetermination may be based on accessing a linked directory source namesfield of the entry of the primary directory file to determine whether atleast one linked directory source name is present. The method branchesto step 562 when the processing module determines that there is not oneor more linked secondary directories (e.g., no linked directory sourcename is present). The method continues to step 556 when the processingmodule determines that there is one or more linked secondarydirectories.

The method continues at step 556 where the processing module accesseseach of the one or more linked secondary directories. The accessingincludes utilizing the at least one linked directory source name toretrieve at least one set of encoded secondary directory slices from theDSN memory, dispersed storage error decoding the at least one set ofencoded secondary directory slices to produce one or more secondarydirectory files, identifying an entry of each secondary directory fileof the one or more secondary directory files corresponding to thesnapshot ID and/or the file name, and extracting the entry of eachsecondary directory file of the one or more secondary directory files.

The method continues at step 558 where the processing module removes asource name reference of the primary directory from each of the linkedsecondary directories. The removing includes deleting the source name ofthe primary directory from a linked directory source names field of eachsecondary directory file of the one or more secondary directory files,dispersed storage error encoding each secondary directory file toproduce one or more sets of encoded secondary directory slices, andstoring the one or more sets of encoded secondary directory slices inthe DSN memory utilizing the at least one linked directory source name.

The method continues at step 560 where the processing module determineswhether there is at least one newer snapshot. The determination may bebased on extracting a snapshot ID entry from a snapshot of each entry ofeach secondary directory file of the one or more secondary directoryfiles and comparing each snapshot ID entry to the snapshot ID of theprimary directory. The processing module determines that there is atleast one newer snapshot when at least one snapshot ID entry is greaterthan the snapshot ID of the primary directory. The method branches tostep 564 when the processing module determines that there is at leastone newer snapshot. The method continues to step 562 when the processingmodule determines that there is not at least one newer snapshot.

The method continues at step 562 where the processing module deletes thedata file associated with the snapshot ID. The deleting includesextracting a source name of the data file from the entry of the primarydirectory file and outputting one or more delete encoded data slicemessages to the DSN memory utilizing the source name of the data filesuch that a plurality of sets of encoded data slices associated with thedata file and the snapshot ID are deleted from the DSN memory.

The method continues at step 564 where the processing module deletes theentry of the primary directory corresponding to the snapshot ID. Thedeleting includes deleting the entry of the primary directory file toproduce a modified primary directory file, dispersed storage errorencoding the modified primary directory file to produce at least one setof encoded modified primary directory slices, and outputting the atleast one set of encoded modified primary directory slices to the DSNmemory for storage therein utilizing the source name of the primarydirectory.

FIG. 18 is a diagram illustrating another example of a directory filestructure that includes directory files 1-2, segment allocation tables(SAT) 1-2, a plurality of data segments 1.11, 1.12 etc., and a pluralityof data segments 2.11, 2.12 etc. Alternatively, any number of directoryfiles, SATs, and data segments may be included. The directory files 1-2may be utilized to affiliate file system filenames to storage locationswithin a dispersed storage network (DSN) memory. The storage locationmay be specified by a source name and/or a vault source name within theDSN memory.

Each directory file of the directory files 1-2 may be stored as encodeddirectory slices in the DSN memory at a location affiliated with thedirectory file. For example, directory file 1 is dispersed storage errorencoded to produce one or more sets of encoded directory 1 slices thatare stored in the DSN memory at location source name 1 (e.g., B530). Asanother example, directory file 2 is dispersed storage error encoded toproduce one or more sets of encoded directory 2 slices that are storedin the DSN memory at location source name 2 (e.g., 42DA).

Each directory file of the directory files 1-2 includes a file namefield 540, a snapshot field 544, an extended data field 546, a linkeddirectory source names field 548, and a SAT source name field 566. Eachfield of the directory file includes one or more entries, wherein eachentry of the one or more entries per field is associated with an entrywithin each other field of a common row of the directory file. The filename field 540 includes one or more entries, wherein each entry of theone or more entries includes a file system file name including at leastone of a root directory name, a directory name, and a file name. Forexample, a directory name entry of the file name field includes /listsand a file name entry of the file name field includes /file.doc and/pic2.jpg.

The snapshot field 544 includes one or more entries, wherein each entrythe one or more entries includes a snapshot ID of a corresponding entry(e.g., same row) in the file name field. For example, a snapshot fieldentry of 1 associated with the file name field entry of /file.docindicates that the file with file name /file.doc is associated with asnapshot ID of 1. As another example, the file with file name /pic2.jpgis associated with a snapshot ID of 2. As yet another example, thedirectory file with directory name /lists is associated with a snapshotID of 5.

The extended data field 546 includes one or more entries, wherein eachentry of the one or more entries includes at least one of a timestamp, asize indicator, metadata, and a content portion associated with acorresponding entry (e.g., same row) in the file name field. Forexample, an extended data field entry of 329 d associated with the filename field entry of /file.doc indicates that the file with file name/file.doc is associated with an extended data value of 329 d. As anotherexample, the file with file name /pic2.jpg is associated with anextended data value of a401. As yet another example, the directory filewith directory name /lists is associated with an extended data value offb79.

The linked directory source names field 548 includes one or moreentries, wherein each entry the one or more entries includes zero ormore source names of linked directory files associated with acorresponding entry (e.g., same row) in the file name field and/or acorresponding entry in the snapshot field. For example, a linkeddirectory source names field entry of 42DA associated with the file namefield entry of /file.doc indicates that the file with file name/file.doc and snapshot ID 1 is associated with a linked directory filewith a DSN address of 42DA. As another example, the file with file name/pic2.jpg and snapshot ID 2 is associated with the linked directory filewith the DSN address of 42DA. As yet another example, the directory filewith directory name /lists is not associated with a linked directoryfile.

The linked directory source name field 548 further provides linkagebetween two or more portions of the directory file structure. Thelinkage may be utilized when directory files include affiliated entries.The affiliation includes entries that share common filenames withdifferent snapshot IDs, entries that share common filenames withdifferent revisions, entries of file names that are moved from a firstdirectory to a second directory, and entries of filenames that arecloned from a first directory to a second directory. For example, asecond revision of file name /pic2.jpg of a second snapshot included indirectory file 1 is linked to a first revision of file name /pic.jpg ofa first snapshot included in directory file 2. As another example, afirst revision of file name /file.doc of a first snapshot included indirectory file 1 is linked to a second revision of file name /file2.docof a second snapshot included in directory file 2.

The SAT source name field 566 includes one or more entries, wherein eachentry of the one or more entries includes a SAT source name of acorresponding entry (e.g., same row) in the file name field. Forexample, a SAT source name field entry of B672 associated with a filename field entry of /file.doc indicates that the file with file name/file.doc is stored in the DSN memory (e.g., as a plurality of sets ofencoded data slices) at a location specified in a SAT 1, wherein SAT 1is stored in the DSN memory at location B672. As another example, a filewith file name /pic2.jpg is stored in the DSN memory at a locationspecified in a SAT, wherein the SAT is stored in the DSN memory atlocation 7AA6. As yet another example, a directory file with directoryname /lists is stored in the DSN memory at a location specified in aSAT, wherein the SAT is stored in the DSN memory at location 90DE.Accessing such a directory file associated with /lists may be utilizedto access one or more files under the directory /lists. For example,accessing the directory file stored in the DSN memory may be utilized toaccess a file associated with a file name of /lists/summary.doc. Asanother example, accessing the directory file stored in the DSN memorymay be utilized to access a sub-directory of /lists/documents andaccessing the sub-directory of /lists/documents may be utilized access afile associated with a file name of /lists/documents/reportA.doc. Assuch, the directory file structure may be associated with any number oflevels (e.g., sub-directories).

Each SAT of SATs 1-2 includes an other data field 568 and a start vaultsource name field 570. Each field of the SAT includes one or moreentries, wherein each entry of the one or more entries per field isassociated with an entry within each other field of a common row of theSAT. The other data field 568 includes one or more entries, wherein eachentry of the one or more entries includes a data segment size indicator,a segmentation approach (e.g., fixed size, ramping size), and a totallength of all segments indicator.

The start vault source name field 570 includes one or more entries,wherein each entry of the one or more entries includes a vault sourcename associated with a first data segment of an associated file. A firstset of encoded data slices corresponding to the first data segment arestored in the DSN memory at a location specified by the vault sourcename. A second set of encoded data slices corresponding to a second datasegment is stored in the DSN memory at a location specified by the vaultsource name plus offset of one. Each successive set of encoded dataslices corresponding to successive data segments is stored in the DSNmemory allocation specified by the vault source name plus a segmentnumber offset (e.g., data segment number−1). A number of successive setsof encoded data slices corresponding to the number of successive datasegments is based on the total length of all data segments indicator ofthe other data entry of the SAT. For example, a first set of encodeddata slices corresponding to a first data segment 1.11 of the file/file.doc is stored in the DSN memory at a vault source name of B673, asecond set of encoded data slices corresponding to a second data segment1.12 of the file /file.doc is stored in the DSN memory at a vault sourcename of B674 (e.g., B673+2−1), etc. until the entire data file stored(e.g., a number of data segments multiplied by the size of each datasegment equals the total length of all data segments indicator).

A request to delete a file may result in deletion of an associateddirectory file entry, deletion of an associated SAT, and deletion ofencoded data slices associated with the file in accordance with adeletion method. The deletion method may be based on one or more of asnapshot ID associated with a file name of the file from a primarydirectory file and one or more associated snapshot IDs and correspondingfilenames from one or more linked directory files (e.g., utilizing oneor more linked directory source names from the primary directory file).

For example, a plurality of encoded data slices associated with filename /file2.doc starting at vault source name B775 are deleted, one ormore sets of encoded SAT slices associated with file name /file2.doc atvault source name B774 are deleted, a plurality of encoded data slicesassociated with file name /file.doc starting at vault source name B673are deleted, one or more sets of encoded SAT slices associated with filename /file.doc at vault source name B672 are deleted, a directory file 2entry associated with file name /file2.doc is deleted, and a directoryfile 1 entry associated with file name /file.doc is deleted when arequest is received to delete the file associated with the file name/file2.doc since file name/file2.doc is associated with a snapshot ID of2, only one linked directory exists (e.g., directory file 1), anassociated entry of linked directory file 1 for file name /file.doc isassociated with a snapshot ID of 1 (e.g., older), and the deletionmethod specifies to delete older snapshots when a newer snapshot isdeleted.

As another example, the directory file 1 entry associated with file name/file.doc is deleted and the one or more sets of encoded SAT slicesassociated with file name /file.doc at vault source name B672 aredeleted, when a request is received to delete the file associated withthe file name /file.doc since file name/file.doc is associated with asnapshot ID of 1, only one linked directory exists (e.g., directory file2), an associated entry of linked directory file 2 for file name/file2.doc is associated with a snapshot ID of 2 (e.g., newer), and thedeletion method specifies to not delete newer snapshots and associatedolder snapshots one and older snapshot is deleted. The method to processa request to delete a file is discussed in greater detail with referenceto FIG. 19.

FIG. 19 is a flowchart illustrating another example of deleting asnapshot, which include similar steps to FIG. 17. The method begins withsteps 550-554 of FIG. 17 where a processing module receives a deletesnapshot request, accesses an entry of a primary directory correspondingto the snapshot identifier (ID), and determines whether there are one ormore linked secondary directories. The method branches to step 572 whenthe processing module determines that there is not one or more linkedsecondary directories (e.g., no linked directory source name ispresent). The method continues to step 556 of FIG. 17 when theprocessing module determines that there is one or more linked secondarydirectories.

The method continues with steps 556-560 of FIG. 17 where the processingmodule accesses each of the one or more linked secondary directories,removes a source name reference of the primary directory from each ofthe linked secondary directories, and determines whether there is atleast one newer snapshot. The method branches to step 564 of FIG. 17when the processing module determines that there is at least one newersnapshot. The method continues to step 572 when the processing moduledetermines that there is not at least one newer snapshot.

The method continues at step 572 where the processing module deletes thedata file associated with the snapshot ID. The deleting includesextracting a segmentation allocation table (SAT) source name from theentry of the primary directory file, retrieving at least one set ofencoded SAT slices based on the SAT source name, dispersed storage errordecoding the at least one set of encoded SAT slices to produce a SAT,extracting a start vault source name of a first data segmentcorresponding to the data file from the SAT, determining a plurality ofvault source names associated with other data segments corresponding tothe data file based on extracting other data from the SAT (e.g., a datasegment size indicator, a total length of all segments indicator), andoutputting one or more delete encoded data slice messages to a dispersedstorage network (DSN) memory utilizing the start vault source name andthe plurality of vault source names such that a plurality of sets ofencoded data slices associated with the data file and the snapshot IDare deleted from the DSN memory.

The method continues with step 564 of FIG. 17 where the processingmodule deletes the entry of the primary directory corresponding to thesnapshot ID and continues at step 574 where the processing moduledeletes a segmentation allocation table associated with the snapshot ID.The deleting includes outputting one or more delete encoded SAT slicemessages to the DSN memory utilizing the SAT source name correspondingto the entry of the primary directory file such that at least one set ofencoded SAT slices associated with the data file and the snapshot ID aredeleted from the DSN memory.

As may be used herein, the terms “substantially” and “approximately”provides an industry-accepted tolerance for its corresponding termand/or relativity between items. Such an industry-accepted toleranceranges from less than one percent to fifty percent and corresponds to,but is not limited to, component values, integrated circuit processvariations, temperature variations, rise and fall times, and/or thermalnoise. Such relativity between items ranges from a difference of a fewpercent to magnitude differences. As may also be used herein, theterm(s) “operably coupled to”, “coupled to”, and/or “coupling” includesdirect coupling between items and/or indirect coupling between items viaan intervening item (e.g., an item includes, but is not limited to, acomponent, an element, a circuit, and/or a module) where, for indirectcoupling, the intervening item does not modify the information of asignal but may adjust its current level, voltage level, and/or powerlevel. As may further be used herein, inferred coupling (i.e., where oneelement is coupled to another element by inference) includes direct andindirect coupling between two items in the same manner as “coupled to”.As may even further be used herein, the term “operable to” or “operablycoupled to” indicates that an item includes one or more of powerconnections, input(s), output(s), etc., to perform, when activated, oneor more its corresponding functions and may further include inferredcoupling to one or more other items. As may still further be usedherein, the term “associated with”, includes direct and/or indirectcoupling of separate items and/or one item being embedded within anotheritem. As may be used herein, the term “compares favorably”, indicatesthat a comparison between two or more items, signals, etc., provides adesired relationship. For example, when the desired relationship is thatsignal 1 has a greater magnitude than signal 2, a favorable comparisonmay be achieved when the magnitude of signal 1 is greater than that ofsignal 2 or when the magnitude of signal 2 is less than that of signal1.

As may also be used herein, the terms “processing module”, “processingcircuit”, and/or “processing unit” may be a single processing device ora plurality of processing devices. Such a processing device may be amicroprocessor, micro-controller, digital signal processor,microcomputer, central processing unit, field programmable gate array,programmable logic device, state machine, logic circuitry, analogcircuitry, digital circuitry, and/or any device that manipulates signals(analog and/or digital) based on hard coding of the circuitry and/oroperational instructions. The processing module, module, processingcircuit, and/or processing unit may be, or further include, memoryand/or an integrated memory element, which may be a single memorydevice, a plurality of memory devices, and/or embedded circuitry ofanother processing module, module, processing circuit, and/or processingunit. Such a memory device may be a read-only memory, random accessmemory, volatile memory, non-volatile memory, static memory, dynamicmemory, flash memory, cache memory, and/or any device that storesdigital information. Note that if the processing module, module,processing circuit, and/or processing unit includes more than oneprocessing device, the processing devices may be centrally located(e.g., directly coupled together via a wired and/or wireless busstructure) or may be distributedly located (e.g., cloud computing viaindirect coupling via a local area network and/or a wide area network).Further note that if the processing module, module, processing circuit,and/or processing unit implements one or more of its functions via astate machine, analog circuitry, digital circuitry, and/or logiccircuitry, the memory and/or memory element storing the correspondingoperational instructions may be embedded within, or external to, thecircuitry comprising the state machine, analog circuitry, digitalcircuitry, and/or logic circuitry. Still further note that, the memoryelement may store, and the processing module, module, processingcircuit, and/or processing unit executes, hard coded and/or operationalinstructions corresponding to at least some of the steps and/orfunctions illustrated in one or more of the Figures. Such a memorydevice or memory element can be included in an article of manufacture.

The present invention has been described above with the aid of methodsteps illustrating the performance of specified functions andrelationships thereof. The boundaries and sequence of these functionalbuilding blocks and method steps have been arbitrarily defined hereinfor convenience of description. Alternate boundaries and sequences canbe defined so long as the specified functions and relationships areappropriately performed. Any such alternate boundaries or sequences arethus within the scope and spirit of the claimed invention. Further, theboundaries of these functional building blocks have been arbitrarilydefined for convenience of description. Alternate boundaries could bedefined as long as the certain significant functions are appropriatelyperformed. Similarly, flow diagram blocks may also have been arbitrarilydefined herein to illustrate certain significant functionality. To theextent used, the flow diagram block boundaries and sequence could havebeen defined otherwise and still perform the certain significantfunctionality. Such alternate definitions of both functional buildingblocks and flow diagram blocks and sequences are thus within the scopeand spirit of the claimed invention. One of average skill in the artwill also recognize that the functional building blocks, and otherillustrative blocks, modules and components herein, can be implementedas illustrated or by discrete components, application specificintegrated circuits, processors executing appropriate software and thelike or any combination thereof.

The present invention may have also been described, at least in part, interms of one or more embodiments. An embodiment of the present inventionis used herein to illustrate the present invention, an aspect thereof, afeature thereof, a concept thereof, and/or an example thereof. Aphysical embodiment of an apparatus, an article of manufacture, amachine, and/or of a process that embodies the present invention mayinclude one or more of the aspects, features, concepts, examples, etc.described with reference to one or more of the embodiments discussedherein. Further, from figure to figure, the embodiments may incorporatethe same or similarly named functions, steps, modules, etc. that may usethe same or different reference numbers and, as such, the functions,steps, modules, etc. may be the same or similar functions, steps,modules, etc. or different ones.

While the transistors in the above described figure(s) is/are shown asfield effect transistors (FETs), as one of ordinary skill in the artwill appreciate, the transistors may be implemented using any type oftransistor structure including, but not limited to, bipolar, metal oxidesemiconductor field effect transistors (MOSFET), N-well transistors,P-well transistors, enhancement mode, depletion mode, and zero voltagethreshold (VT) transistors.

Unless specifically stated to the contra, signals to, from, and/orbetween elements in a figure of any of the figures presented herein maybe analog or digital, continuous time or discrete time, and single-endedor differential. For instance, if a signal path is shown as asingle-ended path, it also represents a differential signal path.Similarly, if a signal path is shown as a differential path, it alsorepresents a single-ended signal path. While one or more particulararchitectures are described herein, other architectures can likewise beimplemented that use one or more data buses not expressly shown, directconnectivity between elements, and/or indirect coupling between otherelements as recognized by one of average skill in the art.

The term “module” is used in the description of the various embodimentsof the present invention. A module includes a processing module, afunctional block, hardware, and/or software stored on memory forperforming one or more functions as may be described herein. Note that,if the module is implemented via hardware, the hardware may operateindependently and/or in conjunction software and/or firmware. As usedherein, a module may contain one or more sub-modules, each of which maybe one or more modules.

While particular combinations of various functions and features of thepresent invention have been expressly described herein, othercombinations of these features and functions are likewise possible. Thepresent invention is not limited by the particular examples disclosedherein and expressly incorporates these other combinations.

1. A method for execution by a dispersed storage processing unit, themethod comprises: receiving a request to store data in dispersed storagenetwork (DSN) memory; determining whether the data is to be appended toexisting data stored in the DSN memory, wherein the existing data isstored in the DSN memory as a set of encoded data slices and wherein theexisting data is encoded using a dispersed storage error coding functionto produce the set of encoded data slices; and when the data is to beappended: encoding, using an append dispersed storage error codingfunction, the data to produce a set of encoded append data slices;generating a set of append commands, wherein an append command of theset of append commands includes an encoded append data slice of the setof encoded append data slices and identity of one of a set of dispersedstorage (DS) units of the DSN memory; and outputting at least a writethreshold number of the set of append commands to at least a writethreshold number of the set of DS units.
 2. The method of claim 1,wherein the determining whether the data is to be appended furthercomprises at least one of: receiving an append instruction with thedata; interpreting account information associated with an issuer of therequest to store the data; and interpreting directory information of atleast one of the data and the existing data.
 3. The method of claim 1,wherein the encoding the existing data using the dispersed storage errorcoding function comprises: arranging data blocks of the existing datainto a plurality of groups of data blocks; and on a group of data blockby group of data block basis, matrix-multiplying the plurality of groupsof data blocks with a dispersed storage error encoding matrix to producea plurality of sets of encoded data slice portions and wherein anencoded data slice of the set of encoded data slices includes an encodeddata slice portion from each of the plurality of sets of encoded dataslice portions.
 4. The method of claim 3, wherein the encoding the datausing the append dispersed storage error coding function comprises:partitioning the data into data blocks based on size of the data blocksof the existing data; arranging the data blocks of the data into asecond plurality of groups of data blocks, wherein a group of datablocks of the second plurality of groups of data blocks includes a samenumber of data blocks as a group of data blocks of the plurality ofgroups of data blocks; and on the group of data block by group of datablock basis, matrix-multiplying the second plurality of groups of datablocks with the dispersed storage error encoding matrix to produce asecond plurality of sets of encoded data slice portions and wherein anencoded append data slice of the set of encoded append data slicesincludes an encoded data slice portion from each of the second pluralityof sets of encoded data slice portions.
 5. The method of claim 1,wherein the append command further comprises at least one of: a slicename of a corresponding one of set of encoded data slices; an offsetindicator; and a column identifier of a slice matrix.
 6. The method ofclaim 1 further comprises: receiving second data for storage in the DSNmemory; determining whether the second data is to be appended toexisting appended data stored in the DSN memory, wherein the existingpending data includes the set of encoded data slices and the set ofencoded append data slices; and when the second data is to be appended:encoding, using the append dispersed storage error coding function, thesecond data to produce a second set of encoded append data slices;generating a second set of append commands; and outputting at least awrite threshold number of the second set of append commands to the atleast a write threshold number of the set of DS units.
 7. A method forexecution by a dispersed storage (DS) unit, the method comprises:storing an encoded data slice of set of encoded data slices, whereinexisting data is encoded using a dispersed storage error coding functionto produce the set of encoded data slices; receiving an append commandthat includes an encoded append data slice of a set of encoded appenddata slices and an instruction to append the encoded append data sliceto the encoded data slice, wherein data is encoded using an appenddispersed storage error coding function to produce the set of encodedappend data slices; and updating storage of the encoded data slice basedon the encoded append data slice to produce an updated encoded dataslice.
 8. The method of claim 7 further comprises: interpreting theappend command to identify a slice name of the encoded append dataslice; identifying the encoded data slice based on the slice name;appending the encoded append data slice to the encoded data slice toproduce the updated encoded data slice; and updating a slice table toassociate the slice name with the updated encoded data slice.
 9. Themethod of claim 7, wherein the updating storage of the encoded dataslice comprises: identifying the encoded data slice; retrieving theencoded data slice to produce a retrieved encoded data slice; appendingthe encoded append data slice to the retrieved encoded data slice toproduce the updated encoded data slice; and storing the updated encodeddata slice.
 10. The method of claim 7 further comprises: receiving asecond append command that includes a second encoded append data sliceof a second set of encoded append data slices and another instruction toappend the second encoded append data slice to the updated encoded dataslice, wherein second data is encoded using the append dispersed storageerror coding function to produce the second set of encoded append dataslices; and updating storage of the updated encoded data slice based onthe second encoded append data slice to produce a second updated encodeddata slice.
 11. A module of a dispersed storage processing unitcomprises: a first module operable to receive a request to store data indispersed storage network (DSN) memory; a second module operable todetermine whether the data is to be appended to existing data stored inthe DSN memory, wherein the existing data is stored in the DSN memory asa set of encoded data slices and wherein the existing data is encodedusing a dispersed storage error coding function to produce the set ofencoded data slices; and when the data is to be appended: a third moduleoperable to encode, using an append dispersed storage error codingfunction, the data to produce a set of encoded append data slices; afourth module operable to generate a set of append commands, wherein anappend command of the set of append commands includes an encoded appenddata slice of the set of encoded append data slices and identity of oneof a set of dispersed storage (DS) units of the DSN memory; and a fifthmodule operable to output at least a write threshold number of the setof append commands to at least a write threshold number of the set of DSunits.
 12. The module of claim 11, wherein the second module functionsto determine whether the data is to be appended by at least one of:receiving an append instruction with the data; interpreting accountinformation associated with an issuer of the request to store the data;and interpreting directory information of at least one of the data andthe existing data.
 13. The module of claim 11, wherein the third moduleis further operable to encode the existing data using the dispersedstorage error coding function by: arranging data blocks of the existingdata into a plurality of groups of data blocks; and on a group of datablock by group of data block basis, matrix-multiplying the plurality ofgroups of data blocks with a dispersed storage error encoding matrix toproduce a plurality of sets of encoded data slice portions and whereinan encoded data slice of the set of encoded data slices includes anencoded data slice portion from each of the plurality of sets of encodeddata slice portions.
 14. The module of claim 13, wherein the thirdmodule is operable to encode the data using the append dispersed storageerror coding function by: partitioning the data into data blocks basedon size of the data blocks of the existing data; arranging the datablocks of the data into a second plurality of groups of data blocks,wherein a group of data blocks of the second plurality of groups of datablocks includes a same number of data blocks as a group of data blocksof the plurality of groups of data blocks; and on the group of datablock by group of data block basis, matrix-multiplying the secondplurality of groups of data blocks with the dispersed storage errorencoding matrix to produce a second plurality of sets of encoded dataslice portions and wherein an encoded append data slice of the set ofencoded append data slices includes an encoded data slice portion fromeach of the second plurality of sets of encoded data slice portions. 15.The module of claim 11, wherein the append command further comprises atleast one of: a slice name of a corresponding one of set of encoded dataslices; an offset indicator; and a column identifier of a slice matrix.16. The module of claim 11 further comprises: the first module isfurther operable to receive second data for storage in the DSN memory;the second module is further operable to determine whether the seconddata is to be appended to existing appended data stored in the DSNmemory, wherein the existing pending data includes the set of encodeddata slices and the set of encoded append data slices; and when thesecond data is to be appended: the third module is further operable toencode, using the append dispersed storage error coding function, thesecond data to produce a second set of encoded append data slices; thefourth module is further operable to generate a second set of appendcommands; and the fifth module is further operable to output at least awrite threshold number of the second set of append commands to the atleast a write threshold number of the set of DS units.
 17. A module of adispersed storage (DS) unit comprises: a first module operable to storean encoded data slice of set of encoded data slices, wherein existingdata is encoded using a dispersed storage error coding function toproduce the set of encoded data slices; a second module operable toreceive an append command that includes an encoded append data slice ofa set of encoded append data slices and an instruction to append theencoded append data slice to the encoded data slice, wherein data isencoded using an append dispersed storage error coding function toproduce the set of encoded append data slices; and a third moduleoperable to update storage of the encoded data slice based on theencoded append data slice to produce an updated encoded data slice. 18.The module of claim 17, the third module is further operable to:interpret the append command to identify a slice name of the encodedappend data slice; identify the encoded data slice based on the slicename; append the encoded append data slice to the encoded data slice toproduce the updated encoded data slice; and update a slice table toassociate the slice name with the updated encoded data slice.
 19. Themodule of claim 17, wherein the third module functions to update storageof the encoded data slice by: identifying the encoded data slice;retrieving the encoded data slice to produce a retrieved encoded dataslice; appending the encoded append data slice to the retrieved encodeddata slice to produce the updated encoded data slice; and storing theupdated encoded data slice.
 20. The module of claim 17 furthercomprises: the second module is further operable to receive a secondappend command that includes a second encoded append data slice of asecond set of encoded append data slices and another instruction toappend the second encoded append data slice to the updated encoded dataslice, wherein second data is encoded using the append dispersed storageerror coding function to produce the second set of encoded append dataslices; and the third module is further operable to update storage ofthe updated encoded data slice based on the second encoded append dataslice to produce a second updated encoded data slice.